System and method for communication with a mobile device via a positioning system including rf communication devices and modulated beacon light sources

ABSTRACT

A light source emits a modulated light, and a radio-frequency transceiver disposed therewith emits a radio-frequency signal. A mobile device may receive either or both signals and determine its position based thereon. The light and radio-frequency sources may be disposed in node in a network of said sources, and the nodes may communicate via the radio-frequency transceivers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of application Ser. No. 15/146,186filed May 4, 2016, which is a Continuation of application Ser. No.14/553,424 filed Nov. 25, 2014, which claims priority to and the benefitof U.S. Provisional Patent Application Ser. No. 61/908,460, filed Nov.25, 2013, and 62/052,207, filed Sep. 18, 2014, which are all herebyincorporated herein by reference in their entirety.

FIELD OF THE INVENTION

In various embodiments, the present invention relates generally tosystems and methods for communicating with a mobile device via apositioning system that includes both radio-frequency (RF) transceiversand modulated beacon light sources, and among the nodes constitutingsuch a system.

BACKGROUND

Provided herein are descriptions of methods and systems for the use andapplication of RF communication devices integrated with light sourcesfor position detection of mobile devices, control of a lighting system,and for communication with mobile and stationary devices, preferably inspaces of relatively limited extent (e.g., retail store interiors,convention floors, libraries, airport terminals, roofed sports stadiums,courtyards) that are illuminated by lights capable of communicatinginformation by visible light communication (VLC). In variousembodiments, these methods and systems enable the RF communicationsdevices to enhance and support the position-determination function andother functions of a system or network of one or more VLC-capablelights. Also in various embodiments, these methods and systemsadvantageously co-locate the RF communications devices with VLC lightspowered by standard mains lighting power, obviating the use of batteriesor the installation of special wiring to power the RF communicationsdevices.

Indoor positioning services relate to systems and methods whereby thelocations of mobile devices and their users within buildings areestimated. Such services may be based on various physical modalities,including light, radio waves, ultrasonic waves, and floor vibrations.Indoor positioning is a key component of location-aware computing, whichrelates to applications that utilize the location of a mobile device toprovide the device's user with content relevant to their location. Somelocation-aware computing functions pertain to entertainment, marketing,and management. An example of an entertainment function supported byindoor positioning is augmented reality, technology that functionallyoverlays a virtual space onto a physical space. Marketing functionssupported by indoor positioning include advertising and contentdistribution through mobile devices (e.g., tablets, cell phones,wearable computers, smart tags). Herein, “marketing” refers to contentdistribution even in nonprofit settings. Indoor marketing functionsdiffer importantly from those for off-site environments: for example,for a retail enterprise, indoor marketing can include unique aspectssuch as alerting a customer via their mobile device of discounts onitems in the store. In an example, a customer not in the store sees anad in general media for a product at a given location. They then travelto that location. Upon entry they are prompted by a radio-frequency (RF)capability of the store's lighting network to view their mobile device.This exposes the mobile device to ambient light and enables visiblelight communication (VLC)-supported position estimation for the device.The customer is then alerted to a special discount coupon, available tothem at that time only, for a product in the store, and may interactwith a map displayed on their device that employs the device's VLCposition estimate and directs them to the product. In general, in-storecontent and ad distribution capabilities are enabled by indoor locationservices that are not feasible in the prior art.

An illustrative management function for indoor positioning is inventorynavigation: e.g., in a warehouse, the indoor positioning system candirect a person equipped with a mobile device to where a particular itemis located. Inventory tracking is also possible: e.g., if the personremoves the item (an event that may be either logged by the device useron their mobile device, or automatically detected as motion of anelectronic tag on the item), a back-end server can update inventorydatabases to reflect this change.

In another exemplary management function, a mobile device using alight-based indoor position positioning technique in conjunction with awireless connection (e.g., WiFi) can enable non-intrusive datacollection on customers. For example, data collection on how customersmove through a store and where they spend time can be collected and usedto improve layout and merchandise displays.

Light-based indoor positioning systems have various advantages oversystems employing other physical modalities. Signals from GlobalPositioning System (GPS) satellites, often used for determining thelocation of mobile computing devices, lose significant power whenpassing through construction materials and suffer from multi-pathpropagation effects, making GPS unsuitable for indoor navigation.Techniques based on received signal strength indication (RSSI) fromradio transceivers (e.g., WiFi and Bluetooth wireless access points)have been explored for indoor positioning, but positioning accuracy ofRSSI-based systems is limited to the multi-meter range. Ultrasonictechniques, which transmit inaudible acoustic waves to microphones, canalso be used to approximate indoor position; however, ultrasound tendsto be more limited in range than radio and commonplace devices andnetworks are not configured for communicating via ultrasound.

VLC indoor positioning techniques employ optical signals, either visibleor infrared, typically emitted by lights also performing an illuminationfunction, and can be used to accurately locate mobile devices indoors.Optical techniques are more accurate and/or reliable than otherapproaches: since light waves are highly directional and do not passthrough most materials, a light-detecting device can be presumedproximate to a light source if the source is robustly detectable.Moreover, many mobile devices are inherently equipped with cameras thatmay be exploited for optical indoor positioning, and are simultaneouslyin contact via radio with one or more communications networks (e.g.,telephone, Internet). Herein, “mobile devices” includes all portabledevices having a computational capability (e.g., phones, tablets,wearable computers, electronic tags, etc.). Also, light-based indoorpositioning systems are easy and low-cost to set up: e.g., to install, abuilding operator may need only to swap out existing bulbs for VLCbulbs. No new socket hardware or power wiring is required, and noscheduled battery maintenance need be instituted.

However, optical indoor positioning techniques have several limitationsor drawbacks. For example, VLC signals cannot be sent or receiveddirectly by many computing devices or networks (e.g., servers, the cellphone network, the Internet); standardized protocols do not exist forthe exchange of data, commands, and other information between VLCbeacons and other computing devices or networks; VLC signals are notdetectable by devices lacking light sensors or whose light sensors arenot exposed to ambient light (e.g., a cell phone stowed in a user'spocket); the bandwidth of VLC beacons is low relative to that of variousother communications modalities; and, because of the tendency of lightto be attenuated by even flimsy objects, any intercommunicating networkof VLC devices (e.g., ceiling lights in a retail space) would likely beunreliable and subject to isolation of portions of the network byarchitectural or other barriers. All of these drawbacks, as well asothers not named, arise from the innate character of VLC signaling.Techniques are therefore desired by which the advantageous aspects ofVLC indoor positioning may be combined with those of one or more othercommunications modalities. Also desired are methods for remote control,monitoring, and diagnosis of a lighting network that may feature non-VLClights and/or VLC lights.

BRIEF DESCRIPTION OF THE FIGURES

The claimed invention and the following detailed description of certainembodiments thereof may be understood by reference to the followingfigures:

FIG. 1 is a representation of a mobile device receiving modulated lightfrom LED light sources.

FIG. 2 illustrates a block diagram of a mobile device exchangingidentification information and location information with a server via anetwork.

FIG. 3 illustrates the process of a mobile device receiving location andcontent information via a light-based positioning system.

FIG. 4 is a process illustrating background services and how theyactivate various sensors contained inside the mobile device.

FIG. 5 illustrates a light-based positioning system in which a clientdevice implements an informational service capability.

FIG. 6 is a high level flow chart of an algorithm for determining tonesembedded within a DPR illuminated area, according to some embodiments ofthe present disclosure.

FIG. 7 is a photograph of a surface under illumination from DPRmodulated signals, according to some embodiments of the presentdisclosure.

FIG. 8 is a post-processed image of a DPR modulated scene afterperforming background subtraction, according to some embodiments of thepresent disclosure.

FIG. 9 is a post-processed image of a DPR-modulated scene after rowaveraging, according to some embodiments of the present disclosure.

FIG. 10 is a plot of the 1-D spectral content of a DPR-modulated image,according to some embodiments of the present disclosure.

FIG. 11 depicts an embodiment of a radio frequency communication enabledlight for transmitting modulated light in accordance with variousembodiments of the invention;

FIG. 12 depicts early detection of a radio frequency identifier tofacilitate detection of a modulated light identifier in accordance withvarious embodiments of the invention;

FIG. 13 depicts a schematic of logical functions and features of a radiofrequency communication enabled light for transmitting modulated lightin accordance with various embodiments of the invention;

FIG. 14A depicts a configuration of RF enabled modulatable beacon lightsarranged in an ad hoc mesh-type network and optionally/temporarilyconnected to Internet resources through a nearby mobile device inaccordance with various embodiments of the invention;

FIG. 14B depicts the logic flow of an illustrative method for self-IDassignment of a light newly installed in a mesh in accordance withvarious embodiments of the invention;

FIG. 15 depicts a flow chart for generating a unique position identifierfrom a combination of a radio frequency identifier and a lightidentifier in accordance with various embodiments of the invention;

FIG. 16 depicts a representative schematic of detection areas of tworadio frequency communication enabled lights for transmitting modulatedlight in accordance with various embodiments of the invention;

FIG. 17 depicts directional transmission of radio frequencycommunication from a light for providing modulated light in accordancewith various embodiments of the invention;

FIG. 18 depicts the generation of a common unique position identifierfrom multiple combinations of radio frequency and light identifiers inaccordance with various embodiments of the invention;

FIG. 19 depicts a flow chart for generating user-targeted content basedon detecting a radio frequency identifier and a modulated lightidentifier from a light in accordance with various embodiments of theinvention;

FIG. 20A depicts an RF capability of a VLC+RF node powered by aconnection to the lighting-conditioned power of a light in accordancewith various embodiments of the invention;

FIG. 20B depicts an RF capability of a VLC+RF node powered by a separateconnection to the internal power supply of a light in accordance withvarious embodiments of the invention;

FIG. 20C depicts an RF capability of a VLC+RF node powered by attachmentto the neutral wire of an AC power source of a light in accordance withvarious embodiments of the invention;

FIG. 20D depicts an RF capability of a VLC+RF node powered by attachmentto the line wire of an AC power source of a light in accordance withvarious embodiments of the invention;

FIG. 21A schematically depicts an illustrative VLC+RF mesh in a firststate of packet transmission in accordance with various embodiments ofthe invention;

FIG. 21B schematically depicts an illustrative VLC+RF mesh in a secondstate of packet transmission in accordance with various embodiments ofthe invention;

FIG. 21C schematically depicts an illustrative VLC+RF mesh in a thirdstate of packet transmission in accordance with various embodiments ofthe invention;

FIG. 21D schematically depicts an illustrative VLC+RF mesh in a fourthstate of packet transmission in accordance with various embodiments ofthe invention;

FIG. 22 depicts the structure of an illustrative digital RF packet inaccordance with various embodiments of the invention;

FIG. 23 schematically depicts a mesh network acting as a transparentrange extender for an RF in accordance with various embodiments of theinvention;

FIG. 24A depicts a mesh network with passively powered tags inaccordance with various embodiments of the invention;

FIG. 24B depicts position estimation of passively powered tags usingreceived radio signal strength indication in accordance with variousembodiments of the invention;

FIG. 25 depicts communication between a modular sensor and a lightingmesh in accordance with various embodiments of the invention;

FIG. 26 depicts the use of tomographic footfall detection by sensorscommunicating with a lighting mesh in accordance with variousembodiments of the invention;

FIG. 27 depicts the use of Angle of Arrival information to assistposition estimation of a mobile device in accordance with variousembodiments of the invention;

FIG. 28 depicts the coordination of the lighting capability of a meshwith user location in accordance with various embodiments of theinvention;

FIG. 29 depicts a use of the coordinated lighting capability of a meshin commission the mesh in accordance with various embodiments of theinvention; and

FIGS. 30A-30C depict location tracking of a user device despite randomchanging of MAC address in accordance with various embodiments of theinvention.

DETAILED DESCRIPTION

In FIGS. 1-10 and discussion thereof, systems and methods are disclosedthat disclose providing a positioning service for devices based on lightreceived from one or more light sources. FIGS. 11-30 pertain to theintegration of VLC and radio capabilities in a lighting system.

In some embodiments, light sources are used to provide an indoorpositioning service to mobile devices. Each light source is given adistinctive identification (ID) code, and database associates each IDcode with specific location data (coordinates). The ID code is broadcastby modulating the visible-light output of the LED light source. Thismodulation occurs at speeds undetectable by the human eye yet detectableby a camera-equipped mobile device as modulation artifact in imagesacquired by the camera. Since robust detection of ID informationbroadcast by a point source of light is feasible only the vicinity ofthe source, the mobile device can be presumed to be in the vicinity of alight whose ID it detects. Detected light IDs can therefore be used toestimate the mobile device's location, e.g., by matching detected IDs tospatial coordinates using a database on a server. When a mobile deviceidentifies light from more than one light source, the position of thedevice can be geometrically estimated with even greater accuracy.

In various embodiments, hashing or other encryption techniques may beused to obscure ID codes from third-party observers (e.g., competitorswho wish to map a store and its VLC system). Herein, “hashing” denotesthe unique mapping of a digital string to a data string of fixed size,with slight differences in input data producing large differences inoutput data. The hashed output for a given input string constitutes anencryption of the input string. For example, in an illustrativeembodiment hashed ID codes may be detected by a mobile device andtransmitted to a server for de-hashing; the hash function itself isnever exposed to third-party examination by being entrusted to themobile device.

FIG. 1 schematically depicts portions of an illustrative embodiment ofindoor VLC positioning in which a mobile device 100 is illuminated byone or more LED light sources. Having detected the IDs of the one ormore light sources, the device 100 communicates (e.g., via a radio meansof communication) with a database that associates light IDs with lightlocation coordinates and, potentially, other content. The location ofthe illustrative embodiment may be any space capable of beingilluminated by VLC light sources in a readily detectable manner.

In the illustrative embodiment of FIG. 1 a mobile device 100 receiveslight 102, 104, 106 from three light sources 108, 110, 112. Each lightsource 108, 110, 112 can be any lighting source used for general-purposeillumination, spot illumination, or backlighting that is capable oftransmitting information by means of visible light communication (VLC).For the purposes of this disclosure, we consider any form of lightcapable of being suitably modulated (e.g., an LED light source) as a VLClight source: LED light sources are referred to extensively herein, butsuch references are illustrative and technologically nonrestrictive.

LEDs can be dimmed and brightened (i.e., modulated) more rapidly thanthe human eye can detect. LED light is therefore a medium through whichdata can be transmitted to devices equipped with appropriatelight-sensing capability, e.g., a digital camera. As will be clear topersons familiar with the art of communications engineering, many formsof digital (two-state) and analog (continuous-state) modulation may beapplied to an LED's light output, including amplitude-shift keying,amplitude modulation, frequency modulation, and many others. Forexample, the brightness of a light source may be modulated by one ormore sinusoidal signals. All modulation techniques that can be appliedto visible light are contemplated and within the scope of the invention,subject only to the condition that the modulation is of a degree and/orfrequency not detectable by a human observer. Moreover, modulation offorms of light not generally detectable by a human observer (e.g.,low-level infrared light), which not constrained by limits on flickerperception, is also contemplated and within the scope of the inventionand intended whenever “VLC” and similar terms are used.

An example of a modulation technique employed in various embodiments isDigital Pulse Recognition (DPR). This technique exploits the rollingshutter mechanism of a complementary metal-oxide-semiconductor (CMOS)image sensor such as is found in many mobile-device cameras. As shall befurther clarified with reference to FIGS. 6-12, a rolling-shutter CMOScamera sequentially captures portions of each image frame, whether astill photograph or a video frame, on a rolling (time-sequential) basis.These portions are typically rows of sensors (e.g., charge-coupleddevices [CCDs]) corresponding to lines of image pixels that are exposedfor successive periods of time. (A CCD may be “exposed” under electroniccontrol—that is, may accumulate light-induced electrical charges—foronly for a fraction of the time that it is actually illuminated by alight source. Herein, “exposure” refers to a period of chargeaccumulation, not a period of illumination by light.) Because rows ofCCDs are exposed sequentially, illumination may be of a differentbrightness during the exposure of each row. Accordingly, a time-varyinglight source may produce row-wise striping in an image frame. Forexample, a light source modulated by a periodic waveform will tend toproduce stripes of even width across a CMOS image, stripe width being afunction of waveform period and rolling-shutter exposure timing. Thefrequencies of one or more distinct VLC tones can be deduced fromstriping patterns in an image.

For example, consider the three light sources 108, 110, 112 of theillustrative VLC beacon light-based indoor positioning system of FIG. 1with the output of lights 108, 110, and 112 sinusoidally modulated at,respectively, 500 Hz, 600 Hz, and 700 Hz. These frequencies are wellabove the human flicker fusion threshold (i.e., about 60 Hz; rarely, ashigh as ˜120 Hz), therefore undetectable to the eye. Mobile device 100within view of lights 108, 110, 112 can detect DPR tones in light 102,104, 106; correlate detected tones with light IDs by accessing adatabase that links IDs to light locations (as shall be shown more fullyin FIG. 2); and then geometrically calculate (or receive the results ofsuch a calculation from a server, not shown in FIG. 1) an estimate ofits own location based on the locations of the detected lights 108, 110,112. Even a single light detection would suffice for an estimate of theposition of device 100.

The illustrative device 100 comprises an image sensor 114, analysismodule 116, and digital memory 118 in order to capture images andanalyze the images to determine if VLC information is present therein.To detect information encoded in ambient light, the mobile device 100can analyze one or more image frames captured by the sensor 114 by usingthe module 116, whose logic can be implemented in any combination ofhardware and software. For example, the module 116 can be an applicationthat runs on a computational capability of the device 100. A networkadapter 120 enables the mobile device 103 to connect to, for example,cellular telephone, Bluetooth, and WiFi networks. The network connection120 can be used to access a data source containing light ID codes linkedto location data and location-dependent content. Linking of ID codes tolocation data can also be accomplished by storing location data in themobile device memory 118, but the network adapter 120 allows for greaterflexibility and decreases the internal device resources needed.

Estimation of position as described above, and/or by other means, canoccur with little to no user input, and can be employed in a number ofways, depending on the desired application. In an illustrative case ofan indoor navigation application, the user would see an identifying “youare here” marker overlaid on a map of the indoor space they are in. Inthe case of content delivery, the user might see on their mobile device100 images, text, videos, recorded audio, or other information aboutobjects they standing near.

FIG. 1 also depicts internal components commonly found in a typical,illustrative LED light source (e.g., light 112) with the addition ofcomponents to allow for the transmission of optical signals via VLC. TheVLC LED light source 112 connects (e.g., via Edison socket or otherstandard socket) to alternating current (AC) electrical power (e.g.,mains power at 120 V, 60 Hz or other standard power) via a converter 122that converts AC power into an direct-current (DC) power appropriate formost electronic devices. Light 112 also comprises a modulator 124 thatincreases and decreases power to one or more LEDs 126 to modulate theirbrightness, a microcontroller 128 which controls the rate and shape ofmodulation of the LEDs 126, and an LED driver circuit 130 which providesappropriate voltage and current to the LEDs 126.

The microcontroller 128 contains a nonvolatile memory storage area(e.g., programmable read only memory, electrically erasable programmableread only memory, or Flash) that stores the ID code of the light 112.The task of the microcontroller 128 is to send a predetermined (e.g.,programmed) sequence of signals to the modulator 124, which in turninterfaces with the LED driver 130 to modulate the visible light fromthe LEDs 126. The sequence of signals sent from the microcontroller 128determines the form of modulation imposed on the light from the LEDs 126and the information conveyed by such modulation.

FIG. 2 is a schematic depiction of an illustrative system 200 in which amobile device 100 that is illuminated by one or more VLC lights (notshown) is connected wirelessly via its network adaptor (e.g., adaptor120 of FIG. 1) to a network 202. A multiplicity of mobile devices (notshown) illuminated by VLC lights may communicate with the network 202,as may other devices and networks (not shown). The network 202 canconsist of a wired or wireless local area network with a method toaccess a larger wide area network or a cellular data network (e.g.,Edge, 3G, 4G, LTS). In essence, the mobile device 100 sends VLC detectedlight-source ID codes 206 through the network 202 to a server 208. Theserver 208 returns location information 204 to the device 100.

In various embodiments, the network 202 may comprise a hierarchy ofnetworks. Such a hierarchy of networks may include a network ofradio-frequency (RF) devices (e.g., Bluetooth low-energy transceivers)that are co-located with the VLC lights in an indoor space, that arepowered from the power source of the VLC lights, and thatintercommunicate with each other in a mesh topology or other networktopology. The VLC lights may be controlled through the nodes of thelow-level RF mesh (e.g., the lights' ID codes, modulation schemes, andon-off behaviors may be thus controlled). Such VLC-integrated RF meshmay relay data between any devices capable of suitable RF communicationwith the nodes of the mesh, including mobile devices, stationarycomputers, sensors, smart tags, robots, drones, and other devices.

Communications from the device 100 to the network 202 can include dataadditional to or other than light ID codes 206, including sensor datasuch as but not limited to GPS coordinates, compass measurements, andaccelerometer/gyroscope data. Communications from the network 202 to thedevice 100 could include data additional to or other than location data,including but not limited to recorded audio, videos, text, and images,and the selection of such data may depend on criteria that include oneor more of (but are not limited to) device location, temporal criteria,historical criteria, and user-specific criteria. Temporal criteria caninclude, for example, time of day, week, month, or year; for example,seasonal marketing and content could be automatically presented to theuser of device 100, or content in the form of morning, evening, ornightly specials could be presented at particular times throughout aday. Historical criteria can include user location history (e.g.,locations visited frequently), Internet browsing history, retailpurchases, or any other information recorded about a mobile device user.User-specific criteria can include age, gender, past buying habits, andother information, as well as policies or rules set up by a user tospecify the type of content they wish to receive or actions the mobiledevice should take based on location information. In an example ofcontent provision based on user-specific criteria, a 20-year-old maleuser may receive different advertisements and promotions than a60-year-old female user.

Illustratively, the capability of the server 208 may be physicallyand/or logically divided into portions or “areas” 210, each of whichperforms functions pertaining to a distinct service location. Distinctservice locations are physical zones, either within a single structureor at diverse locations, that are served by distinct VLC systems. Eachlocation-specific server area 210 contains databases 212 and web-serviceresources 214 devoted to the location. Databases 212 consist essentiallyof information pertaining to a specific service installation (e.g.,light IDs 216, maps 218, content 220, analytics 222). Theinstallation-specific web-service resources 214 enable services whichallow users, customers, administrators, and developers access to IDcodes, indoor locations, and other information.

In an illustrative VLC indoor location service, the server 208 handlesincoming ID codes 224 to appropriately return indoor location data 226to the network for transmission to the mobile device 100. Handling of IDcodes can include receiving incoming ID codes 224, searchingservice-location-specific databases 212 to enable code-to-light matches,calculating position coordinates based on ID codes 224, andcommunicating indoor location data 226.

In a typical VLC-enabled installation, tens to hundreds of LED lightsources broadcast distinctive ID codes. The purpose of the Light IDsdatabase 216 is to record where the ID codes are physically assigned atthe installation. These records can take the form of but are not limitedto GPS coordinates mapped into an indoor space. Maps in alocation-specific Maps database 218 can take various physical anddigital forms, being obtained either directly from the service locationor from third-party vendors or other outside sources. Alocation-specific Content database 220 may contain, for example, ads andinventory information, and a location-specific Analytics database 222may contain, for example, records of user shopping behavior. Databasesmay in general contain static, user-based, and/or dynamic content.Static content is unchanging information associated with the specificarea 210, e.g. audio recordings, streaming or stored video files, texts,images, or links to local or remote websites. User-based content isdependent on user criteria, e.g., user age, sex, preference, habits,etc. Dynamic content changes with varying frequency, e.g., daily,weekly, monthly, etc.

At a typical VLC service location there is a small possibility of havingduplicate ID codes, since there are in practice a finite number ofavailable codes. To deal with duplicate ID codes, additionaldistinguishing information can be contained in the individual logrecords of the Light IDs data base 216, e.g., records of ID codes oflights in physical proximity to each other. The probability of a patternof ID code proximities being inadvertently duplicated at a singlelocation, or at more than one location, can be made vanishingly small byappropriate ID generation and assignment procedures. Distinguishinginformation may also include sensor data, including but not limited toaccelerometer or gyroscope data, WiFi triangulation or fingerprintingdata, GSM signature data, infrared or Bluetooth data, and ultrasonicaudio data.

In various embodiments, “calibration” of a light-based positioningsystem herein refers minimally to the population of the Light IDsdatabase 216 with light IDs installed at a given service location andphysical location information for those lights. In an example,calibration of a light-based positioning system, either upon firstinstallation or after the addition or replacement of VLC lights, isperformed by having a user of a mobile device equipped with anapplication having a calibration capacity walk around the indoor servicespace of the positioning system. The mobile calibration applicationcontains map information for the indoor space, with the positions of theLED light sources overlaid on the map. As the user changes position,they will receive ID codes from various lights. When the user receivesan ID code, they will use the map on the mobile app to select which LEDlight source that they are standing under. Specific lights may be madeto blink (e.g., by commands sent through an integrated Bluetooth mesh)in order to facilitate identification of lights by the user. After theuser confirms the identity of a light, the mobile application sends arequest to the server 208 to match the ID code detected with the lightlocation contained in the lighting plan. In various embodiments, thecalibration may be automated by using robots or flying drones to quarterthe indoor space, associating light IDs with physical locations andpotentially undertaking other calibration-related functions as well,e.g., performing automated, laser-based indoor-surveying operations inorder to produce highly accurate position measurements for associationwith light ID detections. Manual or automated calibration procedures mayalso be employed to acquire finely-space measurements of light fields,radio fields (e.g., radiated by a mesh of Bluetooth transceivers), orother variables throughout the volume of a served space, a processherein termed “fingerprinting.” Thus, calibration minimally includes theassociation of light IDs with light locations, but may also include thegathering of other data, including fingerprinting.

Calibration of LED light locations can also be achieved viacrowd-sourcing. In this method, as mobile devices carried by users movearound an indoor space receiving ID codes 206, they send requests to theserver 208 containing the light ID code 206 received. The server knowsthe device's current approximate position based on other positioningtechniques such as WiFi, GPS, GSM, inertial sensors, etc. Given enoughusers, machine learning algorithms running on the server 208 can be usedto accurately infer the position of each LED light source. The accuracyof this calibration method depends upon several variables, includingnumber of mobile application users. Various computational techniques,including Bayesian techniques, can be used to combine spatialinformation from a variety of sources to produce light locationestimates that are improved or updated at intervals: for example,light-location estimates from a manual or automated commissioningprocess can be made more accurate and complete over time byincorporating crowd-source information in an iterative Bayesian process.For example, a light-based positioning system that was manuallycommissioned at first startup may be subsequently updated bycrowd-sourcing as lights are replaced or added.

FIG. 3 is a schematic depiction, according to an illustrativeembodiment, of the process 300 of receiving location and contentinformation through visible light. Block “User places mobile deviceunder light” 302 corresponds to the act of physically placing acamera-equipped mobile device underneath a VLC-enabled LED light source,e.g., by pulling the device out of a pocket. The user happens to bestanding approximately underneath or adjacent to the LED light source.Block “Sample image sensor” 304 denotes the action, by an app installedon the mobile device, of commanding the embedded digital image sensor ofthe mobile device to acquire an image and reading the image data fromthe sensor. Block “Receive ID?” 306 denotes the conditional action toeither move forward (if one or more light IDs are detected in theimage), or return to “Sample image sensor” 304. Block “Get locationdata” 308 denotes the transmission of the one or more light IDs to aserver and receipt from the server of a location estimate for the mobiledevice. Block “Content?” 310 denotes an inquiry by the server as towhether content (e.g., ads) exists that is associated with the receivedone or more ID codes (and/or the location estimate derived from the oneor more ID codes) and meets criteria for transmission to the mobiledevice app. If transmissible content is available, the procedurecontinues to block “Get content data” 312, where the content is accessedand transmitted to the mobile device; if not, the process ends.

FIG. 4 is a schematic depiction of a method 400, according to anillustrative embodiment, for turning on background services of a VLClocation service application running on a mobile device and fordetermining when to sample an image sensor of the mobile device. Block“Initiate Background Service 1” 402 denotes activation of the firstbackground service running on the mobile device (as, e.g., a capabilityof a VLC app running on the device). Service 1 is tasked withcommunicating wirelessly to determine if the mobile device is close to aVLC-enabled area. The wireless communication may compriseradio-frequency communication techniques such as global position system(GPS), cellular communication (e.g., LTE, CDMA, UMTS, GSM), WiFicommunications, or other. Block “Estimate Position” 404 denotes afunction that samples the wireless communication signal and, based onsampled observations (e.g., RF signal strength measurements, RFtransmitter ID codes) decides whether or not the mobile device is closeenough to a VLC-enabled area to move to the next service.

Block “Light positioning enabled?” 406 denotes a conditional action tomove forward if the mobile device is estimated to be within aVLC-enabled area or to repeat the “Estimate position” function if not.Block “Initiate Background Service 2” 408 is activated if the mobiledevice is estimated to have entered a VLC-enabled enabled area. Service2 is tasked with performing the functions that obtain locationinformation from modulated light. Block “Sample ambient light sensor”412 denotes a function to order the ambient light sensor to beginacquiring images at its inherent frame rate. Block “Detect change?” 412is a conditional action that moves forward if the Service 2 determinesthat the sequential image frame data show a change in observed light.The purpose of this conditional is to determine when the sensor has gonefrom dark to light (e.g., when the user has taken the device out of apocket). Alternatively or additionally to measuring changes in imagecontent, Service 2 can also look for absolute brightness values in imagecontent (obviating the need to compare images) and can test formobile-device accelerometer measurements above a fixed threshold (e.g.,to determine when the user has taken the device out of their pocket). If“Detect change?” produces an affirmative, the method 400 moves to block304 “Sample the image sensor,” after which block 414 “Continue” denotescontinuation of the method of FIG. 3 from block 304.

FIG. 5 schematically depicts, according to an illustrative embodiment,aspects of a light-based positioning system 500 in which a client device502 implements an informational service capability 504 of the system500. The network 202 is a generic local or remote network, or hierarchyof networks, used to connect mobile devices (e.g., mobile device 506) atlocations A 508, B 510, and C 512 with the service capability 504. Eachlocation 508, 510, 512 contains multiple LED light sources (e.g., source514), each of which broadcasts a unique VLC ID code. The client 502 canaccess a database service application 516 that implements multipleclient privilege levels for different levels of access to databases 216,218, 220, 222. These access levels are implemented by correspondingapps. These access apps include User 518, which serves general front-endsystem users; Administrator 520, serving users usually at an IT oroperations management level at an installation; Developer 522, enablingaccess to the application programming interfaces of the system forcustom application development; and Root 524, which enables mastercontrol over other user categories and access to everything contained inthe databases 216, 218, 220, 224.

In typical operation of the system 500, mobile devices in locations 508,510, 512 detect ID codes from lights and send the codes through thenetwork 202 to the Database Service Application 516 through User app518. A mobile device thus obtains read-only access to Maps 218 and(conditional on various criteria administered by the Database ServiceApplication) Content 220. The Database Service Application has built-inwrite access to Analytics 222 in order to record the behaviors of theaccessing mobile device.

In another mode of operation of system 500, the client 502 connects tothe Database Service Application via an appropriate access app 518, 520,522, or 524, enabled via a password-authorized login screen. Clientswith administrator permissions have read/write access to Light IDs 216,read access to maps 218, read/write access to Content 220, and readaccess to Analytics 222. Clients with developer permissions have readaccess to Light IDs 216, read access to Maps 218, read/write access toContent 220, and read access to Analytics 222. A client with rootpermissions has read/write access to all databases. In variousembodiments, a user may access the client device 502 at any permissionslevel through a device communicating with the network 202, e.g., amobile device or a computer at a remote location.

DPR Demodulation Techniques

VLC light sources may produce patterns of stripes in rolling-shutterCMOS camera images. Image processing can be used to extract transmittedsignal from striping. For example, consider a room containing five VLClight sources that are sinusoidally brightness-modulated at 500 Hz, 600Hz, 700 Hz, 800 Hz, and 900 Hz respectively. Each sinusoid, also knownas a DPR tone, can be used to identify its distinctive source (and thusproximity to the source and thus the approximate location of thecamera). Herein, we presume that the exposure parameters of the CMOScamera exposed to light from the five VLC light sources are such as toenable the appearance of unaliased, detectable striping artifact in theresulting images (e.g., the CMOS camera exposure time is not so short asto undersample any of the sinusoidal light signals).

Described herein are two illustrative methods for extracting DPR toneinformation from a rolling-shutter CMOS images. One possibility, when asingle tone is present, is to use line-detection algorithms to identifythe pixel width of the stripes, which directly corresponds to thetransmitted DPR tone frequency. This stripe width is then used to accessa lookup table that associates width and transmitted frequency anddetermines the transmitted tones.

FIG. 6 schematically depicts a second method 600 for performing DPRdemodulation according to one illustrative embodiment, based on imagepre-processing and Fourier frequency analysis (i.e., fast Fouriertransform, “FFT”). Block “Illuminate scene” 602 denotes the illuminationof a scene, e.g. a portion of a store interior, by one or more VLC lightsources and the exposure of a rolling-shutter CMOS image sensor of amobile device to a lighted scene. In this context, “scene” refers to anyarea within view of the camera. Block “Acquire Image(s)” 604 denotesacquisition of one or more images by the image sensor. In the one ormore images, the brightness of any given pixel is partlyscene-dependent, that is, depends on both the details of the scene aswell as on the time-varying VLC illumination. Scene dependence meansthat pixels in the same row of the image (exposed simultaneously, thusnot exposed to different levels of time-varying VLC illumination) willnot all have the same brightness, and also that the relative brightnessof different image rows is not solely dependent on the modulatedillumination. If one calculates the FFT of such an image, the frequencycontent of both the scene and of any DPR tones modulating theillumination will be present.

To recover DPR frequency content independently of the scene, abackground subtraction algorithm may be employed to essentially removethe scene, leaving (ideally) only DPR striping artifact. In an example,an implementation of a background subtraction method uses a videosequence of images. For any DPR tone frequency that is not an exactmultiple of the video frame rate, the DPR bands will shift position fromone frame to another. Therefore, if a sufficient number of video framesare averaged, the banding effect due to periodic modulated modulationwill be reduced to a constant value added to all pixels of the averagedimage. If the video is of a motionless scene, the averaged image willreveal only the underlying scene. This underlying scene (i.e., thebackground) may be subtracted from any individual frame to leave onlythe banding from illumination modulation. The effects of backgroundscenes may also be mitigated, alternatively or additionally, bydeliberate defocusing of the mobile device camera (a form of low-passimage filtering).

For video of a scene in motion, simple averaging of video frames willnot accurately yield the underlying scene background. In such a case,motion compensation is necessary. By using standard feature-recognitiontechniques (e.g., phase correlation) that will be familiar to personsversed in the art of image processing, scene motion between video frames(for example, shifting or rotation of the whole scene due to cameramovement) may be detected, after which each video frame may be shiftedor transformed to cause it to approximately overlie the previous frameas much as possible. After performing these compensatory transforms oneach frame in a series, the resulting frames may be averaged to producea scene background image that may be subtracted from one or more of theframes to reveal DPR striping. In FIG. 6, block “Perform MotionCompensation and Background Subtraction” 606 denotes these operations.

After background subtraction, analysis of an FFT can be used to recoverDPR tone or tones from any one frame or from an averaged frame. As willbe clear to persons familiar with the science of signal processing,given a series of amplitude measurements of a signal containing one ormore periodic components, the FFT of the series will feature peaks atthe frequencies of the periodic components. Automated peak-findingtechniques may be applied to such an FFT to identify the frequencies ofany periodic components. Block “Perform Frequency Analysis to DetermineTone(s)” 608 denotes the performance of such analysis. In FIG. 6, theblock “Return ID(s)” 610 denotes the matching of one or more detectedDPR tones to specific VLC light ID numbers, which can in turn beassociated with physical light locations.

FIG. 7 contains a sample image 700 of a surface illuminated by a lightsource undergoing single-tone (in this case, 700 Hz) sinusoidal DPRmodulation. The image 700 was recorded from a mobile device using arolling shutter CMOS camera. The stripes (e.g., stripe 702) on the image700 are DPR artifact.

FIG. 8 shows an image 800 derived by performing background subtractionon the raw image data of FIG. 7. The result is an image 800 ofalternating dark and light stripes (e.g., stripe 802) that reflectsampling of the transmitted DPR signal. Thanks to backgroundsubtraction, the stripes are more pronounced than in the raw image dataof FIG. 7.

Imperfect background subtraction and/or nonuniform illumination may leadto non-identical pixel values across DPR stripes: this effect is visiblein image 800 as non-uniform brightness along each stripe. Taking FFTs ofrow values along different columns of such an image may producedifferent results, leading to ambiguity. One compensatory method is toassign the average pixel value for any given row to every pixel in thatrow. FIG. 9 displays the image 900 that results from applying rowaveraging to the background-subtracted image 800 of FIG. 8. DPR stripes(e.g., stripe 902) are highly visible and consistent. All columns ofimage 900 contain the same information, i.e., are simply repetitions ofa single, averaged column, so only the averaged column need be retainedfor FFT.

FIG. 10 shows the FFT 1000 of a column from row-averaged image 900 ofFIG. 9. There is a peak at the DPR tone frequency of 700 Hz and a peakat 0 Hz. The 0 Hz (DC) peak does not correspond to a DPR signal; it maybe removed either by low-pass filtering of the amplitude data of FIG. 9or by employing a peak-detection algorithm that discards peaks at ornear 0 Hz.

FIG. 11 depicts an embodiment of a radio frequency communication enabledlighting device 1100 for transmitting modulated light to facilitateposition determination of proximal computing devices, such as mobilephones, tablets, and the like. A modulating beacon light source may becombined with an RF communication capability, such as a Bluetoothcapability or the like, to facilitate combined light and radio frequency(e.g., wireless) signal communication between the light and proximaldevices. Detailed embodiments of a radio frequency communication means(e.g., a WiFi signal function) advantageously combined with a modulatingbeacon light for position determination of proximal mobile devices aredescribed herein.

Modulating light source 1102 may include a processor 1104, such as amicrocontroller, that may be configured to execute programs and processdata that facilitate modulation of light from one or more LED lights1108. The processor 1104 may be connected to a radio frequency (RF)transmit/receive antenna 1110 that may facilitate communication overwireless RF signals to other similarly equipped proximal devices, suchas other modulating light sources, personal mobile devices, computingdevices generally, RF equipped items such as appliances, tools,entertainment devices, RF tags, RF enabled network access points,multi-radio devices, and the like.

The modulating light source 1102 of FIG. 11 may further include one ormore sensors 1112 for detecting aspects of the environment, includingelectromagnetic emissions from nearby computing devices. The sensors1112 may be connected to the processor 1104 to facilitate collection,analysis, and communication of sensor data and/or data derived from thesensor data. Sensors may include ultrasonic sensors, video sensors,audio sensors, image sensors, optical sensors, temperature sensors,humidity sensors, air quality sensors, motion detection sensors,chemical sensors, radio frequency sensors, and the like. While theaforementioned sensor examples are contemplated, so are any other typesof sensor that may detect an aspect of an environment into which themodulating light source 1102 may be deployed.

An RF enabled modulating light source 1100, such as the one depicted inFIG. 11, may facilitate simultaneous Bluetooth transmission and lightmodulation. Position detection is one particular example of anapplication for such an RF enabled modulating light source 1100. In anexample, RF communication between the RF capability of the light 1100and nearby mobile devices may facilitate enhanced position detection, aswill be described herein. Communication among RF enabled modulatinglights 1100 as well as communication between an RF enabled modulatinglight 1100 and a mobile device that is adapted for communication with anRF enabled modulating light 1100 may enable capabilities in positiondetection that may not possible or at least not as efficient withmodulation of light alone. One exemplary capability is bi-directionalcommunication. While position detection via modulating light enablesmobile devices to determine and thereby report their position over anetwork (e.g., WiFi or cellular) to web servers, bi-directional RFcommunication allows the detection of particular mobile devices by thelight 1100. When combined with a potential mesh-type network of RFenabled modulating lights, it is envisioned that a wide variety of data,content, inquiries, promotions, and the like may be effectivelydistributed among mobile devices that have been identified as connectedto at least one RF enabled light 1100.

More generally, a network including or consisting essentially of RFenabled modulating light sources 1100 may enable the control and sensingof numerous aspects of the VLC and non-VLC aspects of the lightingfunction of such a network: any form of control and sensing that wouldbe possible through direct, hard-wired connections to the nodes of sucha network will in general be feasible through the RF aspect of a networkof RF enabled modulating light sources.

RF communication capabilities typically comply with some network-likestandard, such as Bluetooth. As an example, a Bluetooth network standardincludes unique identifiers for each Bluetooth device that is connectedto a network. In a similar way, each RF enabled modulating light 1100may be configured with a unique RF identifier. This RF identifier may beused when determining a position of a properly equipped personal mobiledevice (e.g., a personal mobile device with an RF capability, a cameracapability, and a mobile device application for interacting with atleast these two capabilities). While a capability to receive and processsuch an RF identifier may facilitate coarse device location, it is quitecommon for two or more RF identifiers to be detectable in any of severalpositions due to the natural overlap of RF signals from nearby lights.Therefore RF identifiers alone may not be sufficient for fine positionresolution. However, by combining RF identifier detection withmodulating light detection techniques, improved accuracy and performancemay be achieved.

One such way that improved performance may be achieved is through lightnamespace sharing. Because detection of a light identifier from amodulated light might require repeatedly receiving the modulated lightto detect the specific identifier, when an RF identifier is used incombination with a modulated light identifier, determination of themodulated light identifier may be accelerated. This may be based on aknown relationship between RF identifiers and modulated lightidentifiers, such as through a lookup function that accesses a datasetof identifiers that associates RF identifiers with modulated lightidentifiers. Even if two or more RF identifiers are simultaneouslydetected due to RF signal overlap, determination of the correspondingmodulated light identifier may be much faster because the possiblenumber of modulated light identifiers can be limited to those found inthe lookup function for the simultaneously detected RF identifiers.

FIG. 12 depicts techniques for taking advantage of a wide ranging radiofrequency communication detection area to enhance the performance ofposition detection based on light emissions from a modulated beaconlight 1100. Another benefit of combining RF identification withmodulated light identification involves a capability to pushnotifications to a user of a mobile device that is coming into proximityof a modulated lighting environment due to the generally wider area ofaccess to RF communication signals than modulated light signals.Therefore, detection of an RF identifier transmitted from an RF enabledmodulating light, such as the light 1100 of FIGS. 11 and 12, may enableactivation of a mobile-device function, such as a mobile deviceapplication and the like, to notify the user of his/her proximity to amodulating light for position detection. User notification may includepushing a notification to a user of a mobile device to enable modulatedlight-based position detection when a function on the mobile devicedetects the presence of a transmitted identifier of an RF module that isintegrated into the light. Such notification may include at least one ofan audio and a video notification to the user to expose the mobiledevice's camera so that a modulated light detection application thatuses the device's camera may receive the modulated light.

FIG. 13 depicts logical, functional, and structural features of a radiofrequency communication enabled modulating beacon light source 1100,including a processor 1104, LED light 1108, RF antenna 1110, sensors1112, ID function 1302 that may handle modulated light and RF identifierfunctionality, RF communication features 1304 for communicating over theRF radio signal with other lights 100 and/or mobile devices, LED lightmodulator 1308, power distribution 1310 for providing power to the lightfeatures from the power mains provided to the light through its socketconnection, and a memory 1312 that may be used for program storage, datastorage, and the like. In an example, RF ID function data and Light IDfunction data that may be used by ID function 1302 may be stored in thememory 1312. In another example, sensor data captured via sensors 1112may be stored in the memory 1312. In yet another example, data receivedfrom mobile devices via the RF transceiver (TX/RX) interface may bestored in the memory 1312.

FIG. 14A depicts a configuration of RF enabled modulatable beacon lightsarranged in an ad hoc mesh-type network 1402 and optionally/temporarilyconnected to Internet resources through a nearby mobile device 1418.Modulatable lights may be configured so that a modulated light signalprovided from each light may be distinguished from modulated lightsignals produced by nearby lights, such as by lights in the samebuilding, room, hallway, and the like. When such modulating lights areconfigured with RF capability they may communicate configurationinformation from light to light.

A network capability as depicted in FIG. 14A may include access througha mobile device or other RF enabled device to external (non-mesh)networks. In the example of FIG. 14A, a mobile device that may haveBluetooth and WiFi and/or cellular communication capabilities may act asa gateway for communicating data to/from RF enabled modulatable beaconlights. If the lights are configured into a network, it may be possiblefor an Internet resource, such as a position determination server 1408,to communicate to a networked light 1404 or 1404′ by passing datathrough the Internet 1414 and a network access point, such as a cellulartower 1412 or a WiFi router 1410 that may be in communication with amobile device 1418. If and/or when the mobile device 1418 is inBluetooth radio signal proximity to one of the networked lights, anindirect connection may be made between the server 1408 and a light1404. This may allow communication of information collected from mobiledevices via the RF capability (e.g., identities of devices/users thatpass through the area) by one of the lights to a remote server, such asserver 1408. Although a mobile device 1418 is described as the primarygateway from the light mesh network to a wider area network, any othersuitable network-network gateway may be used. A network-network gatewaymay be configured in close proximity to one of the networked lights thatmay enable communication from at least one of the lights throughnetworks such as Ethernet, PoE, Zigbee, and the like. In variousembodiments, software installed on a mobile device that is in contactwith a non-mesh communications network (e.g., an app voluntarilyinstalled on a smart phone that is connected to the Internet via WiFiand/or to a cell phone network) facilitates the mobile device to act asa network-to-network gateway. In an illustrative embodiment, BLE packetssent by a VLC+RF mesh node to a phone are retransmitted (undecrypted) aspayload via the telephone capability of the phone to a server/controllerassociated with the mesh. The packets may contain any data which themesh is capable of circulating. In another illustrative example, amobile device (e.g., tablet computer) contains both (a) a Zigbeetransceiver capable of communicating with MESH of Zigbee low-powernodes, (b) an RF transceiver capable of communication with the RFcapability of nodes in a VLC+RF mesh, (c) an app enabling thetranslation between the Zigbee informational format and theinformational format of the VLC+RF mesh. In effect, items incommunication with the VLC+RF mesh would be enabled to “appear” asZigbee items to the Zigbee network, and vice versa. In these and variousother embodiments that comprise a “bridging” capability, whether throughmobile devices or dedicated, stationary, multi-radio devices, a VLC+RFmesh may be compatibly combined with a wide range of extant ortraditional wireless/control systems that may coexist with the VLC+RFmesh in a working space or that may be non-collocated with the VLC+RFmesh.

Use of mobile devices as gateways between a VLC+RF system and anothernetwork (e.g., wireless mesh) may be opportunistic: e.g., mobile devicesof customers who have installed an app related to the VLC+RF mesh may beopportunistically enlisted as gateways as the devices move in and out ofthe mesh's working space. Such a gateway function may be used, forexample, to effectively increase the bandwidth of data reporting by meshnodes to a server/controller, since under various conditions packets canbe communicated more quickly through a gateway than through a series ofmesh-node retransmissions as per the illustrative protocol discussedherein with reference to FIGS. 21A-21D. Gateway transmission may be usedalternatively or additionally to transmission through a mesh controllernode connected to a non-mesh network: e.g., upon failure of anexternal-connection node or device, a mesh may still be enabled tocommunicate with a server/controller by means of a gateway function,carrying on its various functions while calling for diagnosis and repairof the failure. The bridging and gateway functions of variousembodiments are exemplified in FIG. 23.

In various embodiments, the position determination server 1408 is ageneral-purpose mesh server and controller (back end) that performsfunctions other than or additional to position determination, issuingcommands to the RF and/or lighting capabilities of one or many networknodes, polling network nodes for information garnered from sensors, andso on. A general-purpose back end may be enabled to understand thelocations, movements, and other aspects of mobile devices and otherassets within the service area of the VLC+RF network mesh. Illustrativecapabilities include inventorying, assisted navigation and realityaugmentation, RF asset tag location tracking, robot and drone tracking,time-of-day-based control, real-time user-tailored control of activeassets (e.g., video displays), security management, routine customerassistance, emergency assistance, ambience adjustment (e.g., music,lighting, and other environmental adjustments in response to sensed userbehaviors), and more. In another example, routine scan (advertising)packet broadcasts from Bluetooth-capable mobile devices are detected bythe RF capability of nodes, enabling a mode of position estimation ofthe mobile device based on received signal strength indication (RSSI)and/or node detection pattern. Such estimates may be combined withestimates based on detection of VLC beacons by a light-sensingcapability of the mobile device, e.g., after the device user is promptedto expose their device to light based on detection of their presence bythe RF mode.

Reference is now made to FIG. 14B. Communication from light to light,and of the light network 1402 with a server 1408 as described above withreference to FIG. 14A, may allow automatic configuration ofidentification data for each light so that when a new or replacementlight is added to an existing ad hoc network of lights, the processor inthe light may communicate via the RF capability with other lights todetermine which light identifiers are already in use and so ensure thatthe new light adopts a distinctive identifier (ID). Automaticconfiguration of a light may facilitate determination by a newly addedlight that it is new to the network and may also facilitate determiningwhich light was most recently added by communicating initializationand/or configuration data over the mesh-like network with the otherlights. This may also facilitate determining a positioning of the lightif the lights are installed in a known order, such as starting at oneend of an aisle and proceeding down the aisle. Similarly, RF IDs may bechanged dynamically yet remain unique within a configuration vialight-to-light RF ID communication via such a network. FIG. 14Bschematically depicts steps of an illustrative method, according to someembodiments, of self-commissioning of a light (i.e., New Light) justadded to an existing network 1402 of one or more lights alreadycommissioned. Upon first power-up 1420, New Light possesses a factorydefault ID (e.g., a string of zeroes) possessed by all new lights, and afirmware pointer causing New Light to undertake Action One 1422:Advertise a query to all nodes in network, asking them to reply with ownID with packets addressed to New Light ID. New Light then performsAction Two 1424: For a time interval T (e.g., one minute), long enoughto give all nodes in a mesh 1402 of maximal size time to reply, listenfor responses from mesh nodes with IDs, and during that interval,collect a list of all IDs delivered. After interval T, New Lightundertakes Action Three 1426: self-assign a random ID other than itsfactory default and all IDs (if any) received, alert the server 1408 ofits new ID, and proceed to Action 1428. In Action 1428, New Lightcommences routine node operation per mesh protocols and/or continues acommissioning process whereby the physical location of New Light isdetermined and stored in the server 1408. Various embodiments may employother steps in a self-commissioning process of a new light, e.g., stepsthat enable (a) automatic determination of new-to-this-mesh status by apreviously used but newly installed light (i.e., a light not possessingfactory settings), (b) automatic determination of First Node in New Meshstatus, and (c) automatic positional determination of a new light (e.g.,“I am in fixture 12, Aisle 3”) based on measurements of radio signalstrength from other nodes in the network 402, as described elsewhereherein.

FIG. 15 depicts exemplary functional steps for using radio frequency andmodulated light identifiers to receive an indication of a position of amobile device relative to a modulating beacon light source. In theembodiment of FIG. 15, a mobile device may receive via a modulated lightcaptured by a mobile device camera an ID for a light. The mobile devicemay also receive an RF ID from an RF capability of a light. These IDsmay be combined and sent to a host, such as a position determinationserver. Alternatively or in addition, the combined IDs may be processedthrough a hash function and the output of the hash function may be sentto a position detection server. The mobile device may receive positioninformation from the server in response to the transmitted combined IDand/or hashed combined ID.

FIG. 16 depicts two radio frequency communication enabled modulatingbeacon light sources that are close enough for the radio frequencycommunication detection areas overlap so that a single mobile device inan overlapping region may need further information to disambiguate whichlight the mobile device is closest to. The narrower modulated lightdetection regions may facilitate disambiguation of a light in position 1from a light in position 2, as in this illustrative a mobile devicecannot simultaneously detect a light in position 1 and a light inposition 2.

FIG. 17 depicts an embodiment of a radio frequency communication enabledmodulated beacon light with directionally limited RF emissions, such asby use of a directional antenna. Directionality of RF transmission maybe achieved by a fixed directional antenna design: e.g., a conformalantenna design may be combined with a lighting fixture reflector thatcombines visible light direction with RF signal direction in a singleunit. Or, a separate directional antenna (e.g., panel antenna) may bemechanically steered by actuators. Or, a phased array antenna (e.g.,antenna employing microelectromechanical systems (MEMS) phase shifters)may be incorporated in a light. Some phased array antennas canprogrammably shape antenna radiation pattern, alter radiation patternorientation, and modulate broadcast intensity; similar flexibilityapplies to signal reception. In various embodiments employing phasedarray antennas, the broadcast power, reception sensitivity, radiationpattern, direction, and other features of an antenna of an RF capabilityof a VLC+RF node may be dynamically reprogrammed through the VLC+RFmesh, or by a local controller in response to changing conditions, or byanother device (e.g., mobile phone). Such programmable antennacharacteristics may also be set to vary throughout a working space,statically or variably, from node to node. If statically programmed, RFantenna characteristics may be chosen, for example, to optimize overallVLC+RF performance over a working space that is oddly shaped,noncontiguous, or occupied by light-blocking, light-reflecting objectsand other obstacles (e.g., partial walls, large metal objects) that maycause VLC and/or RF device-locating performance to vary or degrade frompoint to point. Such adjustments may be set during a commissioning orspace-fingerprinting process, either manually or automatically (e.g., byquartering the space with a small RF-equipped and/or VLC-equipped flyingdrone or crawling robot to characterize system response throughout thespace). If re-settable, system RF characteristics may be altered in aregular manner, or in response to measured characteristics of systemenvironment such as usage patterns, RF interference, or by othercriteria. In sum, the performance of a VLC+RF system may be improved,and its possible functions increased in number, by the inclusion of aconfigurable RF antenna capability.

FIG. 18 depicts a hashing function that facilitates generating a commonunique position identifier for a plurality of combinations of modulatedlight and transmitted radio frequency identifiers. This capabilityprevents third parties from surveying a deployment of modulatable lightsand generating a competing map of the lights. Without the hashingfunction, it may appear that the light and RF IDs being emitted from asingle light are nearly random, thereby defeating any attempt todetermine the unique position identifier for each light.

The number of light and RF IDs and hashes producible by a finite numberof bits is finite by definition. Therefore, given the manufacture ofsufficiently large numbers of VLC+RF nodes, the repetition of possibleidentifiers, both pre- and post-hash, is inevitable. It follows thatwith some probability, non-unique identifiers may, absent potentiallyburdensome tracking techniques, be deployed within a single workingspace or in multiple working spaces. A mobile device that seeks, forexample, to identify what store it is in by detecting a single nodeidentifier may be caused to err by ID repetition. The likelihood of sucherrors can be reduced to an insignificant level by using combinations oftwo or more light and/or RF IDs as position identifiers presumed to beunique. If IDs are randomly assigned and located in an independentmanner, and that the probability of the probability of any singlecombination of a number P IDs being identical to any other singlecombination of P IDs is approximately proportional to p_(ID) ^(P), wherep_(ID) is the probability of an ID repetition occurring in a givenworking space. Thus, for example, if p_(ID)=1×10⁻⁹, then the probabilityof repeating a group of P=4 IDs is p_(ID) ^(P) ∝(1×10⁻⁹)⁴=1×10⁻³⁶, anumber sufficiently small to ignore in real-world applications.

FIG. 19 depicts a system for accessing user-targeted content based onposition information received from a mobile device of the user, wherethe position information is derived from a combination of radiofrequency identification data and modulated beacon light data and thecontent is targeted based on the determined position and an identifierof the user's mobile device that facilitates access to a user profile.

FIG. 20A depicts an illustrative arrangement for powering somecapabilities of a VLC+RF light 2000 according to various embodiments.The light 2000 includes an AC power source 2002 (e.g., mains wiring), alighting power supply 2004, an LED lamp 2006, and an RF transceiver2008. In various embodiments the lamp 2006 may be of another type (e.g.,incandescent), powered arrangements for mechanically directing the lampmay also be included, and the lighting power supply 2004 may includecircuitry for output modulation of the lamp 2006. The AC power source2002 provides a line conductor 2010 and a neutral conductor 2012 to thelight power supply 2004 and the light power supply provides a live wire2014 and ground wire 2016 to the lamp 2006. The lighting power supply2004 may include an energy storage capability (e.g., a battery and/orcapacitor) that enables the lamp 2006, the RF transceiver 2008, or both,along with other devices included within a VLC+RF unit, to continuefunctioning without interruption during power glitches or outages, up toa time determined by the energy storage limits of the power supply 2004.Alternatively or additionally, the RF transceiver 2008 may contain anenergy storage capability. In FIG. 20A, the RF transceiver 2008 ispowered by connection to the live wire 2014 (e.g., by insertion at thelamp installation point of a socket extender that breaks out power forthe RF unit); in FIG. 20B, by connection to the lighting power supplyvia a bus 2016 that may carry combined power and control/data signaling;in FIG. 20C, by connection to the neutral wire of the AC mains (in whichcase the RF transceiver 2008 may contain an internal power supply); andin FIG. 20D, by connection to the line wire of the AC mains.

It is generally advantageous to piggyback the powering of the RFcapability of a VLC+RF light onto the arrangements for powering thelight, as depicted in FIGS. 20A-20D, since in typical working spaces(e.g., retail stores) mains power is architecturally supplied for alighting system regardless of intent to install a VLC or VLC+RF system.In various other embodiments, the RF transceiver may be entirelybattery-powered, or powered by harvesting acoustic, RF, and/or opticalenergy from its environment (e.g., an RF transceiver capability of alight may be powered by a photovoltaic cell illuminated by the lightitself and/or adjacent lights and/or daylighting, enabling the RFtransceiver to be installed simply by attaching it to the ceiling near alight). Without limitation, all such powering arrangements—wired,battery-powered, ambient-energy-powered, or any combination thereof—arecontemplated and within the scope of the invention.

FIGS. 21A-21D schematically depict the principles of packetcommunication between RF nodes of an illustrative VLC+RF mesh 2100 andthe control of lights in the mesh by such communication according tovarious embodiments. The illustrative VLC+RF mesh depicted in FIGS.21A-21D features six nodes. In this illustrative embodiment, each nodemay resemble the radio frequency communication enabled light fortransmitting modulated light 1100 of FIG. 11; that is, each nodecomprises an RF transceiver capability, a light source, and a VLCmodulation capability. In various embodiments, nodes may havecapabilities other than or additional to those of light 1100 in FIG. 11,and may vary in capability within the mesh (e.g., some nodes may possessonly RF communications capabilities while some may have both RF and VLCcapabilities).

In the illustrative embodiment, the RF transceiver capability of thenodes in FIGS. 21A-21D is of the digital packet type; that is, each RFtransceiver broadcasts bits sequentially according to some physicalmodulation scheme, and the transmitted bits constitute groups or“packets” of fixed or, in some realizations, variable length. The bitsof each packet, as shall be clarified in for one illustrative embodimentin FIG. 22, are divided schematically into a string of contiguous fieldsor sub-groups. The packet fields may include information identifying thepacket itself, the transmitter of the packet (e.g., “Node a”), and theintended recipient of the packet (e.g., “all nodes,” “Node c,” “allnodes in Group Two”); commands (e.g., “turn off your light”); datacollected by sensors (e.g., temperature); and other information. Thebroadcast of some packets may be required by an RF interoperabilitystandard, such as the Bluetooth standard; other packets, whileconforming to the standard, may be transmitted optionally, and mayenwrap information in a manner that constitutes a communicationschannel. In broadcasting, required packets (e.g., Bluetoothadvertisement packets) will preferably be given broadcast priority, andoptional packets will be broadcast opportunistically.

The mesh 2100 of FIGS. 21A-21D operates in a waterfall or non-routedfashion: that is, each node transmits packets to all nodes withinphysical range and receives packets from all nodes within range, andrebroadcasts and/or acts upon the packet, subject to certain rules, toall nodes within range. Some of the rules governing responses toreceived packets are as follows:

-   -   1) When not broadcasting a packet, listen for packets. (This        implies alternate broadcasting and listening. In various        embodiments where the RF capability of a light includes at least        one each of an RF transmitter antenna and receiver antenna,        broadcasting and listening may proceed simultaneously.)    -   2) If a packet is detected, extract its identifier (a.k.a.,        packet ID or identifying information).    -   3) Store the packet ID in local memory (e.g., the RAM of light        1100 in FIG. 11). Also store the packet ID if originating the        packet.    -   4) Compare received packet ID to packet IDs already in local        memory, if any.    -   5) If the packet ID matches one found in memory, classify the        received packet as “seen” and “kill” it: that is, do not act        upon any command it contains and do not rebroadcast it.    -   6) If the packet ID does not match any in memory, examine the        packet to see if it contains commands addressed to the receiving        node (e.g., “turn on the light”).    -   7) If the packet does not contain commands addressed to the        receiving node, take no action beyond rebroadcasting the packet.    -   8) If the packet does contain a command addressed to the        receiving node, then “consume” the packet: that is, (a) execute        the command and (b) retransmit the packet if and only if the        packet is also addressed to other nodes, e.g., if the packet is        addressed to a group of nodes.    -   9) If the packet ID does not match any packet IDs found in local        memory and is not consumable by the local node, place the packet        in the broadcast queue (i.e., broadcast or “advertise” the        packet at the earliest opportunity).    -   10) Retire each packet ID in local memory after it has been        there a certain period of time (e.g., 2 seconds). This permits        packet IDs to be re-used after, at minimum, the specified        retirement age times the number of nodes in the mesh. Re-using        of packet IDs is advantageous because (a) otherwise demand for        RAM might become large and (b) packet IDs are specified by a        fixed number of bits in a packet field and are therefore finite        in number, so when unique packet IDs are exhausted one must        either cease broadcasting or re-use IDs.

It will be clear to persons versed in the science of networkcommunications that the above rules are necessary and sufficient toassure that packets first broadcast by any node of the mesh will becommunicated to every other node in the mesh and will cease to berebroadcast by nodes in the mesh after a number of broadcast events notgreater than the number of nodes in the mesh and potentially muchsmaller. It will also be clear that these rules are exemplary and thatin various other embodiments, rule sets differing in various particularsfrom the example given, and likely more extensive, would also enable thepractical operation of a packet-type, unrouted VLC+RF mesh (e.g., a BLEmesh). All such variations are contemplated and within the scope of theinvention, as is the use of network topologies (e.g., bus, ring, star,tree) other than the illustrative mesh topology of FIGS. 21A-21D.

In the illustrative mesh 2100, each node's RF capability includes orconsists essentially of a single antenna that may only transmit orreceive at any given time: a node therefore listens for packets exceptwhen it is transmitting a packet. While transmitting, a mesh may miss(be deaf to) a packet broadcast by one or more other nodes. Althoughthis potentially allows packets to disappear prematurely from the mesh,in various embodiments the VLC+RF mesh may incorporate at least twofeatures that adequately mitigate the problem of packet-dropping: (1)for a given packet to be dropped by a mesh comprising asynchronousnodes, all nodes within receiving range of all other nodes broadcastingthat packet must attempt transmission of that packet (or some other)simultaneously, and this is highly unlikely in an asynchronous mesh; and(2) as described further hereinbelow, a packet definition standard maypermit the inclusion of a response command that requires a receivingnode to transmit a confirmation packet back through the mesh. Failure bya mesh controller node to receive a response packet may triggerpresumption of a packet drop—loss either of the original packet or ofthe confirmation packet—and thus rebroadcast of the original packet.This process may be repeated until a timeout is reached and a networkfailure flag is raised or until packet receipt is confirmed.

In FIGS. 21A-21D, the progress of a single packet through anillustrative VLC+RF mesh 2100 is schematically illustrated. The packet(not shown) is either delivered to Node a 2102 from a device or networkoutside the mesh or is constructed within Node a 2102 by appropriatedevices comprised by the node; or, a packet may originate within anyother node in the mesh 2100. In the case illustrated, Node a 2102broadcasts (advertises) the packet. In FIGS. 21A-21D, receipt of apacket broadcast by a node is indicated by an arrow drawn between thebroadcasting node and the receiving node; e.g., in FIG. 21A, thebroadcast from Node a 2102 is received by Node b 2104, Node c 2106, andNode d 2108. No other nodes in the mesh 2100 are within physicalreceiving range of Node a 2102. At the head of each arrow denotingsuccessful transmission, a mark within the receiving node denotes thefate of packet: (1) A check-mark indicates that the node simplyre-broadcasts the packet, (2) a bull's-eye indicates that the nodeconsumes the packet (which may or may not entail rebroadcasting), and(c) an “X” indicates that the node kills the packet.

In FIG. 21A, Node e 2110, Node f 2112, and Node g 2114 constitute agroup, Group One 2116, within the mesh 2100 (e.g., lights in aparticular section of a retail space). The packet protocol of mesh 2100enables commands be broadcast through the mesh 2100 that are executed byall nodes in Group One 2116 and by no others; nodes in a Group may alsobe individually addressed and controlled. In the state of mesh operationdepicted in FIG. 21A, the light capability of Node e 2110, Node f 2112,and Node g 2114 is On, as indicated by open cones attached to the nodesof Group One 2114 (e.g., cone 2118). The illustrative packet transmittedby Node a 2102 is addressed to the nodes of Group One 2116 and containsa command to turn off the lights.

In FIG. 21A, the broadcast from Node a 2102 has been received by Node b2104, Node c 2106, and Node d 2108. None of these nodes find the packetID in memory, nor find that they are addressed by the packet. Theytherefore rebroadcast the packet, as indicated by the checkmarks in thenode symbols.

FIG. 21B depicts the results of rebroadcast of the packet by Node b2104, Node c 2106, and Node d 2108. In this illustrative mesh, the nodesare presumed to be operating in a non-synchronous manner so that packetcollision (simultaneous broadcast of a packet by more than one node) israre enough to be neglected. Node a 2102 receives the broadcasts of Nodeb 2104, Node c 2106, and Node d 2108; Node b 2104 receives the broadcastof Node c 2106; Node b 2104 and Node d 2108 are too far apart to receiveeach other's broadcasts; Node c 2106 receives the broadcasts of Node b2104 and Node d 2108; and Node d 2108 receives the broadcast of Node c2106. All packets received by these four nodes are killed, because allnodes find the ID of this packet in their “seen” list. Node e 2110 isthe only node in Group One 2116 within receiving range of any of thefirst four nodes in the mesh (a-d) and receives transmissions from Nodeb 2104, Node c 2106, and Node d 2108. In this illustrative case it isposited that Node c 2106 is the first of these three nodes torebroadcast the packet; therefore, Node e 2110 first sees the packet asbroadcast by Node c 2106, places the packet ID in memory, and kills thepackets redundantly received from Node b 2104 and Node d 2108.

Since Node e 2110 is in Group One 2116, to which the packet isaddressed, upon examining the packet Node e finds a command addressed toitself and its fellow group members. Node e 2110 therefore consumes thepacket, i.e., turns off its light and puts the packet on its broadcastqueue. In FIG. 21B, the turning off of the light capability of Node e2110 is represented by blackening in of the cone 2118.

FIG. 21C depicts a subsequent state of operation of the mesh 2100 inwhich Node e 2110 has rebroadcast the packet. This broadcast is receivedby all nodes in the mesh 2100 except Node a, which is too distant toreceive it. The packet is killed by Node b 2104, Node c 2106, and Node d2108, as these have already seen the packet, but it is consumed by Nodef 2112 and Node g 2114, which turn off their lights (as indicated byblackening in of the cones 2120).

FIG. 21D depicts the final stage in the history of the packet, which isrebroadcast by Node f 2112 and Node g 2114. All nodes in Group One 2116receive this broadcast packet but all identify it as “seen” and kill it.Thereafter, the packet is no longer broadcast by any node in thenetwork; also, its intended function (control of the lights in Group One2116) has been accomplished. Similar propagation of various packet typescontaining data, commands, and the like, whether in this illustrativenetwork or in more extensive networks, may be readily envisaged.

Typically, one node in the mesh 2100 is denoted a “mesh controller,”which has unique authority (within a given mesh) to issue commands,receive data, and communicate with a device, mesh, or network outsidethe mesh. The mesh controller node may be a VLC+RF node or anotherBluetooth-capable device (e.g., phone, laptop, server, jointBluetooth+WiFi device). In various embodiments, as is clarified furtherhereinbelow with reference to FIG. 22, a packet may contain a “commandresponse” field that, set to an active state, instructs addressee nodesto transmit packets acknowledging receipt and execution of the commandpacket or returning data to the mesh controller. For example, if Node a2102 of FIGS. 21A-21D were designated a mesh controller, and the packettransmitted in the illustrative packet history included an activecommand response flag, the state of operation of FIG. 21D would befollowed by transmission of response packets by the three nodes of GroupOne 2116, which response packets would contain information confirmingthe dousing of the Group's lights and which would cease to propagatethrough the mesh 2100 only after having been consumed by Node a 2102. Ingeneral, a mesh controller may poll other nodes in a mesh 2100 for arange of information, including the operational state of its variouscapabilities, sensor data, ID data, LED operational history (e.g.,current, voltage), ambient temperature, and the like. Also, a meshcontroller may issue commands changing both RF and VLC identity codesand other operational parameters of nodes or changing the operationalstates of lights and RF transceivers, and may demand confirmation of thereceipt and execution of such commands, and may communicate withdevices, meshes, servers, networks, and the like outside the mesh.

Commands propagated throughout the mesh from a mesh controller mayenable a number of functions, all of which are intended and within thescope of the invention. Without limitation, these include (a) lightbrightness control (on, off, dimming, flashing, VLC message programming,etc.), directional control of lights equipped with mechanicalactivators, retrieval of data from sensors, and retrieval of datacollected by the RF capability of the mesh from mobile devices, tags,other nodes, and the like.

In various embodiments, provision is made for automatic specification ofa mesh controller node, either upon system startup or in the event offailure of the designated mesh controller node; alternatively oradditionally, the mesh may be configured to operate autonomously in oneor more pre-determined default modes in the event that a mesh controllerceases to operate. For example, in the absence of commands from a meshcontroller, the mesh nodes may be programmed to broadcast VLC beacon IDinformation and keep the lights on until further notice, and recordsensor data in RAM in a wraparound fashion that records a moving windowof most-recent readings.

FIG. 22 depicts an illustrative packet structure 2200 for a VLC+RF meshaccording to various embodiments that employ some forms of the Bluetoothstandard for RF communication. Typically, a Bluetooth device is requiredto transmit packets containing constrained, self-identifying content ona regular basis. Other packets may be transmitted optionally by thedevice. In various embodiments, the optional packets, herein termedVLC+RF packets, may be structured to perform the functions of meshcontrol, illumination control, VLC control, tag sensing, sensorreporting, mobile device detection, and other as described elsewhereherein. The following paragraphs describe the structure of VLC+RFpackets employed by the RF aspect of a VLC+RF mesh according to variousembodiments.

Each VLC+RF packet may be 31 bytes long, where a byte is defined as 8bits (an octet). Each packet may be an Advertising packet (i.e., apacket containing identification or command information) or a ScanResponse packet (i.e., a packet containing data solicited by acontroller, including information about receipt of Advertising packets,execution of commands therein, etc.); Advertising and Command Responsepackets differ in function and content, but not in format. Each packetcomprises a Significant Part and a Nonsignificant Part; the SignificantPart may occupy the whole of the packet. Within the Significant Part orpacket payload may be broken down into a collection of AdvertisementData (AD) Structures, each, as indicated in FIG. 22, composed of asingle byte specifying the length of the AD Structure, a single bytespecifying the AD Structure type, and some number of bytes of AD Data.Type identifiers for AD Structures are typically assigned numbers by theBluetooth Special Interest Group.

In a preferred embodiment, the VLC+RF packet contains a single ADStructure that may occupy up to the whole 31 bytes of the packet; thismaximizes the non-overhead payload content of the packet. Thus, of the31 packet bytes, 1 is occupied by the AD Length specification byte, 1 isoccupied by the AD Type specification byte, and up to 29 bytes areoccupied by a Mesh Message. The AD Type specification byte is set to0xFF, the Bluetooth code for “Manufacturer Specific Data.” Todistinguish a Mesh packet from other Manufacturer Specific AdvertisingData packets that may be sent (e.g., to mobile devices in the workingspace of the VLC+RF system), each Mesh Message will begin with a fixedMesh-Specific Sequence (MSS) of 2 bytes, fixed at some arbitrary value(e.g., 0xB1BC).

The remainder of the Mesh Message (up to 27 bytes) consists of encryptedpacket contents. Encryption prevents unauthorized users from takingcontrol of the VLC+RF mesh and its functions (e.g., turning lights onand off) or harvesting information from the mesh. Nodes will decrypt apacket if they determine that it is not on their Seen list, and examineits contents to see if they are an addressee of the packet. Encryptionmay be by a variety of cryptographic techniques: in one embodiment, areversible cryptographic hash is employed. A portion of the encryptedpacket contents (e.g., 2 bytes), possibly up to and including the entireencrypted packet contents is employed as a quasi-unique packetidentifier (“quasi”-unique because although the number of possible M-bitstrings is large for nontrivial M, it cannot be infinite—hence the needfor ID re-use as discussed hereinabove with reference to FIGS. 21A-21D).This packet identifier may be the string employed by nodes foridentifying packets as seen or not-seen, as discussed hereinabove withreference to the illustrative mesh of FIGS. 21A-21D.

The pre-encrypted or decrypted portion of the packet payload consists of(a) an octet of 8 1-bit flags, (b) the 3-byte ID code of the VLC+RF nodeoriginating the packet, (c) the 3-byte ID code of the addressee, whichmay be either a single node or a group of nodes, and (d) a Mesh Payloadof up to 18 bytes length containing Commands and Parameters.

The AD structure of the VLC+RF packet is as follows, where N≦31 is thenumber of bytes in the AD Structure:

AD STRUCTURE MAP Message- Length AD Type Specific Flags SourceDestination Mesh Specifier Specifier Sequence Byte Address AddressPayload value = N 0xFF 0xB1BC flags Node ID Node or Command + 1 byte 1byte 2 bytes 1 byte 3 bytes Group ID Parameters ≦ 3 bytes 20 bytes

The functions of the Flags Byte bits are defined as follows:

FLAGS BYTE MAP Channel Channel Channel Group Response Reserved ReservedReserved ID ID ID bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0

The functions of the Flags Byte bits are further explained as follows:(1) Bit 7, the Group bit is 0 if the destination address field is anindividual node address, or 1 if the destination addressee field denotesa group number (where Group Address 0 denotes indiscriminate broadcast).(2) Bit 6., the Response bit, is 0 if the receiving node should notgenerate a response packet, and 1 if the receiving node should generatea response packet. (3) Bits 0-2, the three-bit Channel ID field, enablethe specification of 8 (=2³) channels. Each “channel” may be reservedfor the use of a particular mesh. Thus, if the physical broadcast andreception spaces of nearby meshes (e.g., on adjacent floors of abuilding) overlap, traffic may still be segregated between the meshes byuse of the Channel ID field. Each node in each mesh is programmed duringsystem commissioning with its own Channel ID, and if a received packetcontains another Channel ID, the node kills the packet.

The Command field and Parameters within the Mesh Payload field may bedesignated as follows: (1) A single byte may be devoted to specifying aCommand; this allows the definition of 256 (=2⁸) distinct Commands,which is ample for the operation of a VLC+RF mesh in variousembodiments. Examples of Command field values include Set MeshController Node ID, Response Packet (signifying that this packet is aresponse packet), Set Light Brightness Level, Blink Light, Set VLCBeacon ID, Device Data Channel (signifying that the packet payloadconsists of data being transmitted on a virtual channel above the packetlayer), and others. (2) Parameters may contain data of any kind,including RF node IDs, VLC node IDs, light brightness level specifiers,portions of multi-packet commands, data for transmission to mobiledevices in the service space, and a variety of other control,commissioning, and communications data. The Parameters field enables theVLC+RF packet protocol to act (as is common for packet protocols) as thebasis for a structure of one or more virtual data channels that conveymessages broken into fragments for transmission in packets. The bit rateof such virtual channels must always be less than the physical bit rateof the packet layer, but there are few constraints on the informationalcharacter of such virtual channels, which may include field definitions,error correction, encryption, packet structure, and any other featurescapable of embodiment in a digital data stream.

Packet life cycle has already been partly described with reference toFIGS. 21A-21D. In various embodiments, two numerical parameters aredefined for the whole RF capability of the mesh (e.g., programmed intothe nonvolatile memory of all RF nodes) during commissioning of themesh, namely (1) ADV_PER_PKT and (2) ADV_SEEN_TIME_LIMIT. ADV_PER_PKT isthe number of times a mesh packet will be advertised by each mesh node(e.g., once), if the packet is to be advertised. ADV_SEEN_TIME_LIMIT isthe number of seconds after receipt (e.g., two) that a packet ID expires(i.e., is deleted from the packet ID stack in node memory). ADV_PER_PKTconstrains the amount of traffic across the mesh that will be entailedby the transmission of each packet; ADV_SEEN_TIME_LIMIT tends toconstrain the lifetime of a packet in the mesh (although transmissionthrough node rings and other cases, for sufficiently large meshes, couldenable packets to ring through the mesh even after ADV_SEEN_TIME_LIMIThas expired). Decreasing either or both parameters tends to decrease theprobability that a packet will be dropped; increasing either or bothparameters tends to increase the lifetime of each packet in the mesh andso limit the effective bandwidth of the mesh as a whole.

It will be clear to a person familiar with the science of networkcommunications that the foregoing packet and mesh specifications may invarious embodiments be varied in many particulars without substantivelyaltering the capabilities and applications of a VLC+RF described herein.

FIG. 23 depicts the illustrative employment of a VLC+RF mesh 2300 as acommunications range extender or “digital wire” for mobile and otherdevices (e.g., a phone 2302) in the service area of the mesh, accordingto various embodiments. As noted above with reference to FIG. 22, thepacket level of a VLC+RF mesh 2300 may be used to support one or morevirtual channels. Given appropriate design of such a channel, accordingto principles and methods that will be familiar to persons acquaintedwith the science of communications, a VLC+RF mesh 2300 may therefore beemployed as a digital wire or range extender that enables an RF-equippeddevice (e.g., a Bluetooth-equipped mobile device, such as phone 2302) inthe vicinity of the mesh 2300 to communicate with a distant device(e.g., a personal computer 2304, server 2306, or other device 2308)through the mesh 2300 without any necessity for the RF-equipped device2302 to implement the mesh packet protocol. For example, aBluetooth-equipped mobile telephone may establish contact, followingnormal Bluetooth protocol steps, with the BLE capability of a node(e.g., Node A 2310) in the VLC+RF mesh 2300. Some or all nodes in themesh 2300, including Node A 2310, are equipped to recognize andbroadcast non-mesh Bluetooth packets. Upon receiving a non-mesh packetfrom the mobile device 2302, Node A 2310 wraps the non-mesh packet'scontent into one or more mesh packets and broadcasts the one or morepackets using, in an illustrative embodiment, the packet protocoldescribed above with reference to FIG. 22 and addressing the packet orpackets to the mesh controller node 2312. The packet propagates to themesh controller node 2312, which in this example is in communicationwith the Internet 2314 via a WiFi link (i.e., the mesh controller nodeis a multi-radio device). The mesh controller node 2312 unwraps themobile device's Bluetooth packet content from the mesh packet andtransmits it to the Internet 2314, which may in turn communicate withother devices 2304, 2306, 2308. Any response addressed to the mobiledevice 2302 reaches the mesh controller 2312, which wraps the responsecontent into one or more mesh packets and broadcasts the one or morepackets across the mesh 2300 to Node A 2310 (or to all nodes in Node A'sgroup, or to all nodes in the mesh 2300) which unwrap the packet andbroadcast its contents as a standard Bluetooth packet for possiblereceipt by the mobile device 2302. In other illustrative scenarios, themobile Bluetooth-equipped device 2302 communicates with anotherBluetooth-equipped device (not shown) in the working space, or inanother mesh, or elsewhere transparently through the VLC+RF mesh 2300.In brief, a mobile device 2302 need not know that the mesh 2300 isthere, yet the VLC+RF system transparently supports communications ofthe mobile device 2302 with other devices.

Further embodiments, features, and capabilities are described below.

In various embodiments, powering of RF tags and other devices in theworking space of the VLC+RF system may be achieved by harvesting of RFenergy from RF transmissions of the system, from illumination providedby the system, or from acoustic energy provided by the working areaenvironment and/or by an acoustic (e.g., ultrasonic) capability of theVLC+RF system. Recent technological developments have increasinglyenabled the harvesting of very small quantities of electromagnetic ormechanical energy from environments. For example, all receiving antennaeintrinsically collect broadcast electromagnetic energy, which may beutilized for its information content, its energy content, or both. Also,piezoelectric and electromechanical devices (e.g., piezoelectric filmsand microelectromechanical devices) can convert acoustic energy toelectrical energy, which likewise may provide information, energy, orboth. Photoelectric devices can harvest energy from light for eithersensing or power. One or more of these energy-harvesting modalities maybe employed in devices. In various embodiments, a VLC+RF system powersone or more energy harvesting devices (EHDs) in its working area bybroadcasting one or more of (a) ordinary Bluetooth signals in anondirectional manner, (b) Bluetooth signals in a directional manner,e.g., as described hereinabove with reference to FIG. 7, (c)non-communicative RF signals matched to the receiving capabilities ofEHDs, (d) ultrasonic signals matched to the matched to the receivingcapabilities of EHDs, (e) VLC illumination and/or noncommunicativeillumination, and (f) infrared VLC and/or noncommunicative illuminationin a manner matched to the receiving capabilities of EHDs. RF poweringof EHDs is preferred because RF waves, unlike light and ultrasound,readily pass through most objects and materials commonly found in theintended working spaces of such a system; moreover, RF direction ofenergy (e.g., by phased array beam steering or mechanical beam steering,as described hereinabove with reference to FIG. 7) may allow moreenergetically higher-efficiency powering of EHDs whose locations areknown. EHDs may be stationary or mobile, and without limitation mayinclude RF product tags, sensors, and LEDs.

Reference is now made to FIG. 24A, which schematically depicts anillustrative system 2400 in which a lighting mesh 2402 features acapability for communication with radio-powered EHDs, e.g., a tag 2404.Radio broadcasts from or more nodes of the mesh 2402 may be received byone or more tags 2404. Each tag 2404, in this illustrative embodiment,includes or consists essentially of an antenna 2406, an energy-storingcapability (e.g., a capacitor) 2408, and an information processingcapability 2410. The tag antenna 2406 is capable of both receiving radiowaves, whether from nodes of the network 2402 or from other sources, andof transmitting radio waves, as symbolized by the two sets of ripplelines 2412 associated with the antenna 2406. When the antenna 2406receives radio waves, some of which may be transmitted by one or morenodes in the mesh 2402, some of the energy in those waves is stored inthe energy-storing capability 2408. When a sufficient amount of energyhas been stored by the storage capability 2408, the informationprocessing capability 2410 may be powered for some interval. Itsfunctions may include recording data received through the antenna 2406and transmitting information (e.g., an ID code of the tag 2404) throughthe antenna 2406. Transmissions by tags may be received by nodes in thenetwork 2402, enabling location detection of tags and/or by other tags.In various embodiments, tags may in general perform all the functions ofcontinuously-powered nodes in the main network 2402, as well astag-specific functions, enabling the tags to function as apassively-powered secondary network, or extension of the continuouslypowered network 2402. EHDs of various capabilities (not shown) may bepresent within the system 2400. In various other embodiments, EHDs mayharvest forms of energy additional to or other than radio waves; radiowaves may be spatially directed toward EHDs by nodes of the network2402; and other capabilities may be included or omitted.

A technique for estimating the positions of tags, nodes, and othercomponents in communication with a lighting mesh and that may be enabledby RF capable modulating beacon lights is RF signal-based triangulationto determine an approximate position of a mobile device. Thisapproximate position may be further narrowed through the use of themodulated beacon light data from lights in proximity to the approximateposition. In various embodiments, RF position determination may employreceived signal strength indicator (RSSI), a measurement of the powerpresent in a received radio signal, to determine the location of anRF-capable mobile device. Since RF energy per unit area received from anomnidirectional transmitter such as most mobile devices employ drops bythe inverse square of distance, RF transceivers closer to the cell phonewill measure stronger RSSI for the advertising packets. Alternatively oradditionally, a mobile device may make RSSI measurements of broadcastsfrom one or more RF capable modulating beacon lights, which measurementsmay then be used to estimate the device's position by triangulation,possibly taking into account non-omnidirectionality of RF transceiverbroadcasts. Either the raw RSSI measurements or a position estimatecalculated by the mobile device itself from those measurements may bereported by the device to a server, back end, or controller through theRF capability of the VLC+RF mesh or via another channel with which themobile device is in contact (e.g., a cell phone network). In anotherillustrative embodiment, BLE advertising packets broadcast by a cellphone are detected by one or more RF transceivers of a VLC+RF mesh. TheRF transceivers are preferably located in a common plane (e.g., theceiling of a store): however, RSSI may be used for mobile deviceposition estimation even if RF transceiver locations are non-coplanar iftheir three-dimensional locations are accurately known. Thesemeasurements may be reported through the mesh, or via the mobile deviceemployed as a gateway, or via one or more other mobile devices or nodesemployed as gateways, to a mesh controller node or computer thatcontains software capable of algebraically estimating transmitterlocation from the RSSI measurements. RSSI measurements at a singlemoment, or over a short interval, may be used by the VLC+RF mesh andassociated computing devices for position estimation of broadcastingdevice. Repeated position estimates by such means enable the estimationof transmitter dynamics (velocity, acceleration) and movement history.Doppler measurements at a single moment, or over a short interval, mayalternatively or additionally be used in estimating device dynamics.

RSSI measurements, similarly reported to and analyzed by a computerwithin the mesh or in communication with it, may also be used by RFtransceivers of a VLC+RF mesh to estimate their own relative positions.In a preferred embodiment, the RF transceivers are located in a commonplane (e.g., the ceiling of a store), increasing the likelihood of aunique solution for the location of a multiplicity of RF transceiversgiven their mutual observations of RSSI. In various embodiments, RFtransceivers may transmit packet or non-packet signals during acommissioning process or other process of net geometry determinationbased on mutual RSSI measurements: the broadcast of such signals (e.g.,packets, pings, sinusoids, ramped sinusoids) may be controlled on anode-by-node basis through the VLC+RF mesh in a manner that increasesthe informational content of a set of RSSI measurements. The results ofan RF net geometry determination may be combined computationally withthe results of a visible-light net geometry determination based on,e.g., results of “fingerprinting” measurements of light beaconintensity, RSSI, or other measurable features of wave fields within theworking space. The results of an RF and/or VLC net geometrydetermination may also be used, in a directed-type network, to set uprouting tables for network nodes.

FIG. 24B is a schematic depiction of tag location estimation using RSSIin the illustrative system 2400 of FIG. 24A. In FIG. 24B, a tag 2414 inthe working space transmits a packet containing a tag ID. The ID may bedistinctive to the particular tag, or to a class of tags, or have othersignificance. The radio waves transmitted by the tag 2414 will travel afirst distance 2416 to a first node 2418, a second (shorter) distance2420 to a second node 2422, and a third distance 2424 to a third node2426. The nodes 2418, 2422, 2426 are arranged in a non-collinear fashion(not in a straight line) and are equipped with an RSSI detectioncapability. RSSI magnitude measured by each node will depend on thedistance from the transmitting tag to the node. Presuming that all tagstransmit with equal signal strength, all tags at distance 2420 from node2422 (indicated by a dashed circle centered on the antenna of node 2422)will be measured at a unique RSSI by node 2422. Similarly, a given RSSIuniquely determines tag radius for nodes 2418 and 2426 (also indicatedby dashed circles, shown in part and centered on their respectivenodes). There is only one point of intersection for the three RSSIradii, namely, the location of the antenna of tag 2414. Thus, RSSImeasurements by three or more non-collinear nodes may be used tocalculate the position of a node. In various embodiments, angle-ofarrival information from two or more nodes may be used alternatively oradditionally to calculate tag location. In various embodiments, tags mayuse RSSI measurements of pulses from multiple network nodes to calculatetheir own position, and/or to supplement position calculations by meshnodes.

Several geometrical constraints are now noted. Two spheres whose centersare separated by a distance greater than the difference of their radiiand not greater than the sum of their radii intersect along a circle.Two RSSI (i.e., radii) measurements therefore do not suffice todisambiguate the location of a mobile device: the mobile device might beanywhere on a circle determined by the two measured distances and thelocations of the nodes measuring them. Moreover, three or more sphereshaving collinear centers can always be positioned so that all intersecton a circle. Thus, even when three or more radii are measured, collinearRSSI-measuring nodes can produce ambiguous location estimates for amobile device. Finally, all arrangements of intersecting spheres havingcoplanar centers intersect at points, or along circles, that aresymmetrical above and below the plane. There is thus always ambiguity inlocation estimates derived from a coplanar set of RF nodes about whetherthe detected transmitter is above or below the plane. All suchambiguities may be avoided in various embodiments by providing that asystem using RSSI measurements for location estimation comprises atleast four RF nodes, one of which is non-coplanar with the others. Forexample, in a system comprising a three or more coplanar nodes in aceiling, one or more out-of-plane Disambiguation Nodes can also beinstalled. Each such node will always be more proximate to or distantfrom a mobile device that is below or above the ceiling plane, thuseliminating ambiguity.

FIG. 25 is a schematic depiction of tag location estimation in theillustrative system 2400 of FIG. 24. In FIG. 25, an environmental sensor2500 in the working space includes or consists essentially of an antennacapability 2502, an energy storage capability (e.g., a battery) 2504, aninformation processing capability 2506, and one or more sensors 2508which may be modular in nature (e.g., may be plugged in to customize thecapabilities of the sensor 2500) and may sense properties such as sound,nonacoustic vibration, light, RF signals, gasses, and other. Threemodular sensors 2508 are depicted as generic transducers in theillustrative sensor 2500 of FIG. 25. Multiple sensors may be present andmay communicate with the mesh 2402. In various embodiments sensors maybe powered by batteries, harvested environmental energy, or both, andmay also function as tags, and position determination of sensors may beperformed as described above with reference to “tags.”

RF position estimation of passive (EHD) tags and/or sensors linked toassets—e.g., merchandise, robots, furniture, clothing—within a workingspace maybe be used by the VLC+RF mesh for asset position mapping,inventorying, theft detection, and related purposes. Alternatively oradditionally, battery-powered sensors or asset-linked tags maycommunicate with the mesh. In an illustrative embodiment, the VLC+RFsystem is used as a backbone network for network tracking as follows:EHD or battery-powered modules comprising an RF capability regularlybroadcast BLE packets that (a) contain encrypted data based on sensorsvariously comprised by modules such as temperature, humidity, CO or CO₂or other gas levels, occupancy, RF spectrum monitoring, audio recording,brightness sensing, floor vibration, and other; and that, (b) whenreceived by one or more VLC+RF nodes, enable the location of modulethrough RSSI triangulation on the module. In this example, a genericbase module having an RF capability allows for the physical plug-in ofmultiple sensor modules for customization. The sensing network—i.e.,modules plus VLC+RF mesh—is managed via a cloud-based system accessedthrough the web via a representational state transfer (REST) applicationprogramming interface (API). Asset tracking by a back end may be enabledby communication between the mesh and the tag modules. Firmware updatingand other control of tag modules may also be managed through the mesh.Such a system may enable simultaneous asset spatial tracking,inventorying, space usage characterization, environmentalcharacterization, theft detection, and other data-driven functions

FIG. 26 is a schematic depiction of portions of an illustrative system2600 in which sensors are employed to support position estimation forusers of a working space. Sensors 2602, 2604, 2606 are in contact withthe floor of space served by a lighting mesh network 2608. Vibrations2610 from footfalls 2612 transmitted through the floor (plane of thefigure)—potentially from more than one walker, or from a wheelchair inmotion, or from other forms of human or machine motion—may be sensed byappropriate vibration transducers of the sensors 2602, 2604, 2606.Sensor information may be transmitted by the sensors 2602, 2604, 2605 tothe mesh 2608 and thence to a back end or computing device (not shown)capable of estimating the location of the footfalls 2612 throughtomographic techniques (e.g., techniques similar to those known and usedin the science of seismic tomography). User location information fromfloor-vibration tomography may be used alternatively to, or to enhance,location user location information derived from mobile devices carriedby users as described elsewhere herein.

A single microcontroller in a VLC+RF node may be programmed to performall light modulation and RF communication capabilities using timedroutines or functions. The microcontroller may be configured toconstantly broadcast and receive an RF signal while also ensuring thatthe modulation of the light (e.g., on/off state) is properly operated soas to provide the right modulation light function while avoiding humanvisible flicker in the light output.

An alternate configuration may include a separate in-line RFcommunication module that may be disposed in-line with a modulatingbeacon light. This combination may allow each physical combination of RFcapability and modulating light capability to achieve capabilities thatare similar to the integrated embodiment described, such as in FIG. 11above.

Various embodiments of the present invention incorporate RF devices(e.g., the Broadcom BCM4358 802.11ac WiFi 2×2 Multiple Input MultipleOutput chip, and similar devices capable of BLE transceiving) thatintrinsically determine Angle of Arrival (AOA) information for receivedRF signals. Such information may be used, independently of VLC and RSSIinformation, to estimate the position of a mobile device. In anillustrative case, such estimation is performed as follows:

1) Upon reception of a signal from the user's mobile device, the RFcapability of a VLC+RF node measures AOA of the signal and sends thatinformation, along with the node's own distinctive ID information, backto the mobile device via the standard BLE protocol.

2) The mobile device accumulates (and time-correlates, based onreception time) these Angle-of-Arrival+ID data sets from one, two, ormore nodes in the lighting system.

3) The mobile device computes a location estimate from at least twoAngle-of-Arrival+ID data sets transmitted to the mobile device by meshnodes. For example, the device may group the three (closest in time) AOAmeasurements from distinct nodes and use these angles, along with theknown locations of the nodes as determined from their IDs, to form atwo- or three-dimensional position estimate of the mobile device. In anillustrative two-dimensional (planar) case, if two nodes have measuredan AOA of a broadcast from a mobile device, one may draw a line passingthrough each node at the measured angle, whereupon the intersection ofthe two lines estimates the location of the mobile device. A similarapproach can be used for three-dimensional position estimates using AOAmeasurements and known node locations from three or more nodes.

4) A position estimate based on AOA information may be augmented bypositional estimation using RSSI measurements, VLC beaconing, and othersources of positional information (e.g., inertial measurements). Forexample, it will be clear to persons familiar with the science ofprobability and statistics that Bayes's Theorem may be used to make anoptimal position estimate based on multiple sources of positionalinformation. In a Bayesian approach, the position estimate is a 2D or 3Drandom variable having some prior distribution (distribution ofuncertainty prior to the receipt of first data or additional data), withnew AOA, RSSI, and VLC positional information constituting “evidence”that is used to update the prior. The posterior distribution (updatedprior) of mobile device position may then be used as a priordistribution for a new, iterative round of estimation as additionalevidence becomes available. Such an approach may maximize the likelyaccuracy of position estimates in a position estimation system gatheringpositional information from multiple modalities. In various otherembodiments, non-Bayesian methods of combining positional informationfrom two or more sources, or from a time series of measurements, areemployed.

In another illustrative embodiment, which may be implementedalternatively or additionally to the case just described, a mobiledevice measures AOA of packets broadcast by two or more RF nodes in aVLC+RF mesh and receives distinctive ID information from those nodes.Physical node locations derived from the ID information, combined withthe AOA measurements, enable the mobile device to mathematicallyestimate its own location. In another illustrative embodiment, a mobiledevice measures AOA and IDs for node broadcasts, while nodessimultaneously measure AOA and ID for the mobile device, and all thesedata are combined algorithmically, whether by a computational capabilityof the nodes, a node controller, the mobile device, or another device,into a single position estimate for the mobile device. Such an estimatemay be combined with positional estimate information from othermodalities. Moreover, as discussed hereinabove for RSSI measurements,nodes in a mesh may use mutual AOA measurements in order to produce oraugment information about mesh node geometry. Such information may beused to assist or automate commissioning or updating of a mesh map,routing tables, etc.

FIG. 27 is an schematic depiction of an illustrative method of positionestimation of a tag 2700, similar to tag 2404 of FIG. 24, using an AOAcapability of two nodes 2702, 2704 of known location within a lightingmesh (not otherwise shown), or of tag 2700, or of both. In one mode ofoperation, a signal or series of signal (e.g., packet) is transmitted bythe tag 2700 and arrives at the nodes 2702, 2704. The AOA of the tagsignal at node 2702 is A1 (depicted in FIG. 27) and the AOA of the tagsignal at node 2704 is A2. Angle A1 defines a line 2706 on which thenode 2702 and the tag 2700 both lie; similarly, angle A2 defines a line2708 on which the node 2704 and the tag 2700 both lie. The intersectionof the lines 2706, 2708 approximates the location of the tag 2700.Alternatively or additionally, the nodes 2702, 2704 emit pulses that aredetected by tag 2700, which possesses an AOA capability. By the law ofopposite interior angles, tag 2700 also measures AOAs of A1 and A2 andmay therefore calculate (or transmit to a server the AOA informationfrom which may be calculated) lines 2706, 2708 and their intersection,i.e., the location of tag 2700. AOA measurements made within areasonably close time interval by all three devices—nodes 2702, 2704 andtag 2700—may be combined to produce a more accurate position estimate.In various other scenarios, more than one node and more than one tag maysend and receive signals used for AOA position estimation. In variousembodiments, the role of the tag 2700 of FIG. 2700 may be played by asensor, mesh node, or any other device whose signals are capable of AOAdetection by mesh nodes, or which can sense signals emitted by meshnodes and possesses an AOA capability of its own. Tags, sensors, meshnodes, and other intercommunicating devices in the lighting system mayall, potentially, collect and share information that contributes to theposition estimation of space users and physical assets.

Coordination of light function may be accomplished via RF enabledmodulatable lights. This may allow for certain lights to temporarilyproduce a particular modulation signal, emit a particular color, and thelike. In a retail environment, such a temporary use may be used toindicate a special offer or otherwise bring a shopper's attention to aparticular position in the retail space. Likewise, information, offers,content, and the like could be transmitted from RF enabled lights tonearby mobile devices, thereby effectively providing in-storenotification of certain specials, and the like without requiring aformal site-wide customer accessible WiFi network or relying on externalprivate wide area network use (e.g. cellular networks).

FIG. 28 is a schematic depiction of an illustrative system 2800including or consisting essentially of a lighting mesh 2802 that has acoordination of lighting function capability. Mesh 2802 senses thepresence of a mobile device 2804 and approximates its location (e.g., byVLC, RSSI, or another technique as described herein). The back end (notshown) of the mesh determines that the device user 2806 is in thevicinity of a display screen 2808. Through the mobile device 2804, theback end pushes to the user 2806 an invitation to view content on thescreen 2808. The user 2806 indicates interest, whereupon the light 2812emitted by the light 2810 most proximate to the user 2806 is dimmed by acommand transmitted through the mesh and directed specifically to thelight 2812. (In FIG. 28, undimmed light is indicated by a whitetrapezoid under a light, and dimmed light is indicated by across-hatched trapezoid under a light.) The mesh also transmits a signal2814 to the display screen indicating that it is to display certaincontent appropriate to the user 2806. This is an illustrative of manysimilar employments of a network of RF enabled position determininglights having coordination of lighting function capability that maydescribed.

A network of RF enabled position determining lights may enablelight-to-light coordination. Functions such as synchronized modulation,phase locked modulation, duplicated modulation signal broadcasting tostrengthen a particular modulation ID signal, simultaneous modulation toreduce destructive interference, and the like may be implemented.Control of any one light or a group of lights may help with troubleconditions, such as directing emergency personnel to a position in alarge warehouse type store to assist a person in need of medical help.

An application, such as a smart phone application, may be used tofacilitate configuring and optionally controlling a lighting setup for alocation. To prevent random programmers from taking control of thelighting setup, security measures, such as encryption and multi-factorauthentication may be used. Such an application may depict a visual mapof a lighting setup so that accessing (e.g. via a touch screen) any ofthe lights in the map may allow the user to control the selected light(e.g. cause it to visibly blink, turn off, turn on, stop broadcasting amodulation signal, run a diagnostic or setup function, and the like).

Modulated light position determination may be combined with RF positiondetermination in an integrated LED-based light and user-specific profiledata to inform a content server of a position of a nearby mobiledevice/user. In an illustrative embodiment, the content server may thendeliver user-targeted content to a nearby screen (e.g. advertisingscreen) that is proximal to the determined position of the mobiledevice/user. Alternatively or additionally, the content server maydeliver user-targeted content to other media (e.g., speakers) proximalto the determined position of the mobile device/user or to one or moremobile devices of the user. Position determination may be used to directpersonnel to the assistance of a user in the working space: e.g.,personnel may be equipped with mobile devices that inform them of areal-time, mesh-enabled position estimate of a mobile device user whohas requested personal attention, and so be enabled to rendezvous withthe user even if the user's position changes after the personnel aredispatched.

Bluetooth networking capabilities that facilitate rapid delivery acrossa network of a high priority information packet may be exploited in anetwork of RF enabled modulating beacon lights to ensure rapid deliveryof content (e.g., advertisements) from a server through the light-basedRF network to a user's mobile device that is connected to thelight-based RF network.

Commissioning of a newly-installed or revised lighting network may befacilitated by capabilities of a VLC+RF mesh. For example, in acommissioning process where a commissioner carrying a mobile deviceassigns location information to specific ceiling lights, and has justassigned a location to a light, the commissioner may wish to identify anext light for commissioning. However, a plethora of lights may becandidates. Via the RF capability of a VLC+RF mesh, the commissioner mayrequest that candidate lights in their vicinity and proximate to thelight just commissioned be made to blink. By means of RSSI measurementsreported through the mesh to a back end, proximity of candidate lightsto the light just commissioned can be determined, narrowing the field ofcandidates and speeding the commissioning process. FIG. 29 is aschematic depiction of aspects of a commissioning process of a lightingmesh 2900. The locations of some or all of the nodes of the mesh 2900are not initially known. A commissioner 2902 carries a mobile device2904 and wishes to associate a physical location with a next light inthe mesh (e.g., the light 2906). Having previously commissioned a nearbynode 2908, the commissioner 2902 sends a command (through, e.g., an RFcapability of their mobile device 2904) to flash lights in the vicinityof the nearby node 2908. By means of RSSI measurements of node 2908 andpossibly other nodes, the back end of the mesh knows that light 2906 isnear the node 2908 and causes the light 2906 to flash (indicated bystriped trapezoid under light 2906). The commissioner then knows whichlight is the next to commission, and determines a physical location forthe light (e.g., using indoor survey equipment). The physical locationof light 2906 is then associated by the back end of the mesh with thatlight and the commissioner may proceed to associate physical locationdata with other lights in the mesh. In various operating scenarios andvarious embodiments, other procedures may be followed for lightcommissioning assisted by an RSSI capability and other RF capabilitiesof lights.

In various RF communications systems, such as some standards proposedfor cell phones, IDs broadcast by mobile devices are changed randomly toprevent unwanted location monitoring (surveillance) of mobile deviceusers. However, such random periodic ID shifting may defeat locationtracking desired by a user, e.g., a user who has installed anenterprise-specific app enabling their device to communicate with anin-store location detection system. The RF capability of a VLC+RF meshmay enable location tracking of a user despite random ID shifting. Forexample, a BLE-capable mobile device with random periodic ID shiftingwill advertise a given ID a number of times before changing its ID.Pairing one or more spatially correlated location and movement estimates(VLC, RF, or both) with a series of RF ID receptions may enable theprobabilistic (e.g., Bayesian) assignment of a consistent deviceidentity to a transmitter despite random periodic ID shifting. Suchposition tracking will tend to be made more accurate by VLC-based,higher-resolution position estimates of the mobile device, typicallydependent on software voluntarily installed on the user's device andtherefore mitigative of privacy concerns. Location-based services andinformation may therefore continue to be delivered to a customerparticipating in a location tracking system.

FIGS. 30A, 30B, and 30C schematically depict an illustrative use oflocation tracking to permit continuous track of a user whose mobiledevice periodically and randomly changes its media access control (MAC)address. FIG. 30A depicts a first operational state of a mesh 3000 at aTime 1, at which the position of a device 3002 broadcasting a certainMAC address, MAC Address 1, has been estimated by the back end (notdepicted) of mesh 3000. Around the position estimate of Time 1 a circleof radius R may be defined. FIG. 30B depicts a second operational stateof the mesh 3000 at a Time 2, some specific interval after Time 1. InFIG. 30B, in the time elapsed between Time 1 and Time 2 the device 3000has changed its MAC address to MAC Address 2. A device detection andposition estimate at Time 2 by the mesh 3000 finds that the devicebroadcasting MAC Address 2 is less than radius R from the Time 1estimate of the position of device 3002, which broadcast MAC Address 1.Consequently, the back end of mesh 3000 re-identifies device 3002 as thebroadcaster of MAC Address 2. FIG. 30C depicts an occasion on which adevice broadcasting MAC Address 1 is not detected within radius R of theTime 1 location estimate for device 3002; rather, a device 3004broadcasting MAC Address 2 is detected farther than radius R from theTime 1 location estimate for device 2002. Consequently, the back end ofsystem 3000 does not identify device 3004, the broadcaster of MACAddress 2, as the original device 3002; correctly or not, the back endidentifies the broadcaster 3004 of MAC address 2 (in the case of FIG.30C) as a distinct device from device 3002 of FIG. 30A. In variousembodiments, the procedure used to identify device continuity withchanging MAC address or other broadcast identifier may differ in itsdetails; e.g., a more computation-intensive, Bayesian, multi-data-sourcemethod may be employed to estimate broadcaster identities.

Techniques for the broadcast and detection of at least locally uniqueidentifiers for VLC beacon sources are shown and described in, forexample, U.S. Ser. No. 13/369,144, filed on Feb. 8, 2012, U.S. Ser. No.13/369,147, filed on Feb. 8, 2012, U.S. Ser. No. 13/422,591, filed onMar. 16, 2012, U.S. Ser. No. 13/435,448, filed on Mar. 30, 2012, U.S.Ser. No. 13/445,019, filed on Apr. 12, 2012, U.S. Ser. No. 13/446,506,filed on Apr. 13, 2012, U.S. Ser. No. 13/446,520, filed on Apr. 13,2012, and U.S. Ser. No. 13/526,781, filed on Jun. 19, 2012, the entiredisclosure of each of which is incorporated by reference herein. Methodsfor the broadcast and detection of at least locally unique identifiersfor RF transceivers are shown and described herein.

Hashing may include a unique mapping of a digital string to a datastring of fixed size. A networked server may be configured to generatethe unique location identifier from the first and second plurality ofidentifiers. The mobile device may transmit the first and secondplurality of identifiers to the networked sever. A networked server maybe configured to receive the unique location identifier from the mobiledevice. The unique location identifier may be unique to one light or iscommon to a plurality of lights.

Herein, wireless RF communication is on occasion denoted by reference to“Bluetooth,” but any wireless communication means may be similarlyemployed and all such are implicitly intended, subject to constraintsarising from variations in capability, wherever “Bluetooth” and relatedterms are used. Examples of wireless RF communication means intendedherein, without limitation, include Bluetooth, Bluetooth-Low-Energy(BLE), WiFi, Enhanced NFC, WiMax, 3G, 4G, and the like. Also herein,lighting systems described as embodying “VLC” and similar capabilitiesmay minimally possess only an illumination capability, not necessarily avisible-light communications capability, in some or all lights.

Also herein, the term “positioning system” and similar terms referinclusively to systems that perform functions additional or alternativeto determining the position of a mobile device in a working space.Examples of such additional or alternative functions, withoutlimitation, include determining the orientation and/or state of motionof a mobile device in a served space; gathering and/or exchanginginformation about the distribution of magnetic fields, RF signals,illumination, or other measurable quantities throughout a working space;communicating operational commands, sensed data, queries, maps,identifying information, marketing and sales information, and otherinformation with, between, or through mobile or stationary devices inthe working space; wirelessly supplying electrical power to stationaryor mobile devices in the working space; and using stationary or mobiledevices as gateways through which to access other devices or networks,such as an intranet or the Internet. Also herein, the term “indoor”(e.g., in “indoor positioning system”) is understood to include allrelatively local areas or volumes, whether partly or wholly containedinside a building or not, that may be served by a VLC+RF positioningsystem.

Having described one embodiment of the invention, it will be apparent tothose of ordinary skill in the art that other embodiments incorporatingthe concepts disclosed herein may be used without departing from thespirit and scope of the invention. The described embodiments are to beconsidered in all respects as only illustrative and not restrictive.

What is claimed is:
 1. A mobile device, comprising: a radio frequencytransceiver; a hardware user interface; a camera configured to receive avisual light communication; and a processor configured to: receive aradio frequency identifier of a lighting device via the radio frequencytransceiver; receive via the camera a modulated light identifier of alighting device in the vicinity of the mobile device; combine the radiofrequency identifier and the modulated light identifier to form acombined lighting device identifier; and receive position informationrelated to the mobile device based on the combined identifier.
 2. Themobile device of claim 1, wherein the processor is further configuredto: send an indication of the combined lighting device identifier to ahost processor; and receive from the host processor an indication of theposition information related to the mobile device.
 3. The mobile deviceof claim 1, wherein the processor is further configured to: process thecombined identifier with a hash function; and send an output of the hashfunction to a host processor as the indication of the combinedidentifier.
 4. The mobile device of claim 1, wherein the processor isfurther configured to: process the combined identifier with a hashfunction; and send an output of the hash function with the indication ofthe combined identifier to a host processor.
 5. The mobile device ofclaim 1, wherein the received radio frequency identifier is a firstradio frequency identifier, and the processor is further configured to:receive a second radio frequency identifier from another lighting devicevia the radio frequency transceiver; determine that the modulated lightidentifier received by the camera corresponds to the first radiofrequency identifier; and in response to the determination, determine aposition of the mobile device as being in the vicinity to the lightingdevice from which the first radio frequency identifier was received. 6.A method, comprising: receiving an identifier of a lighting device via aradio frequency transceiver of a mobile device; receiving at a camera ofthe mobile device an optical identifier of a lighting device; combiningthe lighting device identifier and the optical identifier to form acombined identifier of the light device in a vicinity of the mobiledevice; sending the combined identifier to a host processor; andreceiving position information based on the combined identifier sent tothe host processor.
 7. The method of claim 6, further comprising:processing the combined identifier with a hash function; and sending anoutput of the hash function to the host processor as the combinedidentifier.
 8. The method of claim 6, further comprising: processing thecombined identifier with a hash function; and sending an output of thehash function with the combined identifier to the host processor.
 9. Asystem, comprising: a first lighting device comprising: a first radiofrequency transceiver; a first light source configured to output amodulated light signal unique to the first lighting device, the firstlighting device configured to: transmit a first radio frequencyidentifier identifying the first lighting device, via the first radiofrequency transceiver in a first radio frequency detection area, andemit a first modulated light signal from the first light source, in afirst modulated light detection region; a second lighting devicecomprising: a second radio frequency transceiver; a second light sourceconfigured to output a second modulated light signal unique to thesecond lighting device, the second lighting device configured to:transmit a second radio frequency identifier identifying the secondlighting device, via the radio frequency transceiver in a second radiofrequency detection area, and emit a second modulated light signal fromthe second light source in a second modulated light detection region,wherein: the first radio frequency detection area and the second radiofrequency detection area overlap one another; and the first modulatedlight detection region and the second light detection region arenon-overlapping; and a mobile device, comprising: a processor, a radiofrequency transceiver; and a camera, the mobile device configured to:receive the first radio frequency identifier via the mobile radiofrequency transceiver, identifying the first lighting device; receivethe second radio frequency identifier via the mobile device radiofrequency transceiver identifying the second lighting device identifier;receive via the camera either the first or the second modulated lightsignal; and in response to receiving a first modulated light signal bythe camera, the first radio frequency identifier, and the second radiofrequency identifier, determine that the mobile device is in a positioncloser to the first lighting device than to the second lighting device.10. The system of claim 9, wherein: the first lighting device furthercomprises a first antenna, wherein the first antenna is configured toemit a signal including the first lighting device radio frequencyidentifier in a first area proximate to the first lighting device; andthe second lighting device further comprises a second antenna, whereinthe second antenna is configured to emit a signal including the secondlighting device radio frequency identifier in a second area proximate tothe second lighting device, wherein the first area and second areaoverlap.
 11. The system of claim 9, wherein the mobile device is furtherconfigured to: prior to determining that the mobile device is in aposition closer to the first lighting device than to the second lightingdevice, combine the first modulated light signal with the first radiofrequency identifier to form a combined identifier; and use the combinedidentifier in the determination that the mobile device is in a positioncloser to the first lighting device than to the second lighting device.